If your legal AI makes a “judgment,” it’s not just following rules.

It’s expressing a worldview.

Most software operates on if/then logic. Input X produces Output Y. Every time. Deterministic.

LLMs are fundamentally different. They're probabilistic, and new research shows just how much that matters.

A recent paper, "Evaluative Fingerprints" by Wajid N., studied 9 frontier LLMs evaluating the same content using the same rubric. The results are striking:

Inter-model agreement was near zero. Yet individual models were remarkably consistent with themselves, just not with each other.

The researchers could identify which model produced an evaluation with 89.9% accuracy based solely on its scoring patterns. Even GPT-4.1 and GPT-5.2 (same provider, different versions) were distinguishable 99.6% of the time.

The paper calls this the "reliability paradox": models don't agree on what "good" means, but they're so consistent in how they disagree that their evaluation patterns function as fingerprints.

Why should legal care?

In many ways, LLMs behave like people, who are also probabilistic. We've always known that different lawyers assess risk differently, interpret contract language differently, prioritize issues differently.

Now we're deploying AI with the same characteristics.

Consider a legal department implementing an agentic workflow that escalates matters based on risk assessment.

This research suggests the choice of model isn't an implementation detail.

It's a substantive decision that shapes outcomes.

As we build agentic processes that assert judgment (contract review, risk triage, compliance monitoring), we need to understand that model selection is a methodological choice with real consequences.

The question isn't whether to use AI for legal judgment.

It's whether we understand what theory of judgment we're encoding when we pick a model.