From model performance to system evidence.

Capable and dependable are different claims

A benchmark score answers a bounded question: given this input, did the system produce the expected output. It is a good question, and it is a portable one, which is why it dominates. A score can be compared across vendors, reproduced by a third party, and quoted in a procurement document without anyone involved having to understand the buyer’s business.

Dependability does not travel that way. To say that an organization can depend on a system is to make a claim about a specific deployment: this model, behind this retrieval layer, with these tools, these permissions, this data, these people and these consequences. Almost none of that claim transfers to another company, and a surprising amount of it fails to transfer to the same company six months later.

FENORI VIEWThe field mostly reports the first and is mostly asked for the second. We do not think the mismatch comes from dishonesty. It comes from the fact that only one of the two produces a number that can be printed on a page and still mean something to a stranger.

What follows looks at three things: what happens to reported results when the unit of analysis moves outward from the model to the system in its environment, why those results should be read carefully, and where the standards bodies are taking the problem.

What the evidence shows once the unit is the system

FINDINGJha and colleagues built ITBench at IBM Research to put agents in front of real information technology operations work rather than exam questions, across site reliability engineering, security and compliance, and financial operations. In the ICML 2025 camera ready, covering 102 scenarios, agents built on state of the art models resolved 11.4 percent of the SRE scenarios, 25.2 percent of the CISO scenarios and 25.8 percent of the FinOps scenarios excluding anomaly detection, with an F1 of 0.35 on FinOps anomaly detection.[1]

SOURCEAn earlier release of the same benchmark, in February 2025, covered 94 scenarios and reported 13.8 percent on SRE, 25.2 percent on CISO and 0 percent on FinOps. Those figures are superseded by the camera ready quoted above. We record them because both sets are still in circulation, and a reader who finds either should be able to see that we knew about the other.[1]

FINDINGEnterpriseBench, published at EMNLP 2025, comes at the same problem from the opposite direction. Rather than sampling real incidents it constructs a sandboxed company, with 500 tasks spanning software engineering, human resources, finance and administrative domains. The environment is built to reproduce the conditions that make enterprise work difficult rather than to remove them: data source fragmentation, access control hierarchies that determine who may see what, and workflows that cross functions. The most capable models completed only 41.8 percent of the tasks.[2]

FENORI VIEWTwo benchmarks built by different groups on opposite principles land in the same region, and it is a long way below where the same class of model sits on knowledge and reasoning tests. We do not read that as evidence that the models are weak. We read it as evidence that the environment carries a large share of the difficulty, and the environment is precisely what a model level benchmark holds constant in order to be comparable.

FINDINGKapoor and Narayanan and colleagues at Princeton approach the same gap from the measurement side. Towards a Science of AI Agent Reliability, at ICML 2026, decomposes reliability into four dimensions grounded in safety-critical engineering practice: consistency, robustness, predictability and safety. Applying those metrics to 12 frontier models across two complementary benchmarks, the work reports that reliability improved at roughly half the rate of accuracy on the general agentic benchmark, and at about one seventh the rate on the customer service benchmark. Agents remain inconsistent across runs, brittle to prompt rephrasings, and poorly calibrated in self-assessment, even as accuracy improves.[3]

FENORI VIEWA ratio of one to seven is the number in this paper we find hardest to argue away. If accuracy and reliability improve at rates that far apart, then the two are not proxies for each other and a headline accuracy figure cannot stand in for both. Selecting a system on that figure is selecting on one property while declining to find out what happened to the other, and the gap between them appears to be widening rather than closing as models get better.

The measurement is itself contested

Everything above rests on measurement, and measurement is where the literature stops agreeing with itself. This section presents the disagreement rather than resolving it, because it is not resolved.

FINDINGSCORE, by Chatterjee and colleagues at the NAACL 2025 Industry Track, held the underlying question fixed and varied only its surface. Under semantically equivalent paraphrased prompts, accuracy fluctuated by up to 10 percentage points on MMLU-Pro. Reordering the answer choices produced differences of up to 6.1 points on AGIEval. The work also evaluated consistency under non greedy inference across five runs.[4]

The Princeton work quoted earlier reports the same brittleness one level up, at the agent rather than the answer: agents remain brittle to prompt rephrasings even as their accuracy improves.[3] Two independent lines of work, one on single answers and one on multi-step agents, observe the same thing.

FINDINGErrica and colleagues, at NAACL 2025, propose measuring that behavior directly instead of treating it as noise. What Did I Do Wrong? introduces two metrics: sensitivity, which measures how predictions change across rephrasings of a prompt and requires no ground truth labels, and consistency, which measures how predictions vary across rephrasings for elements of the same class. They report that the two give complementary insight beyond accuracy, and that they help identify hard samples and problematic classes.[5]

FENORI VIEWA metric that needs no ground truth labels is worth noticing for a practical reason the paper does not have to argue: it can be run against production traffic, where labels do not exist and never will.

FINDINGThe obvious reading of all three results is that the models are fragile. Hua, Tang, Gu, Gu, Wong and Qin argue at EMNLP 2025 that a substantial part of the fragility belongs to the instrument instead. Evaluating seven language models across six benchmarks with twelve prompt templates, Flaw or Artifact? finds that much observed prompt sensitivity stems from heuristic evaluation methods, including log likelihood scoring and rigid answer matching, which overlook semantically correct responses phrased differently. Under evaluation by a language model acting as judge, they observe substantially reduced performance variance and consistently higher correlation in model rankings across prompts. They conclude that models may be more robust to prompt templates than previously believed, and that prompt sensitivity may be more an artifact of evaluation than a flaw in the models.[6]

That is a serious claim and it deserves to be read as one. It does not say the variance was imagined. It says that a specific and very widespread scoring apparatus generates part of it, and that changing the apparatus makes much of it go away. It also leaves something open that its own method cannot close: a language model acting as judge is a model, with sensitivities of its own, so replacing one scorer with another changes the measurement rather than removing the need to justify it. Both sides of this can be right about their own experiments while the general question stays open.

FENORI VIEWWe are not going to pretend this disagreement resolves in the direction that suits us. What we take from it is narrower than either paper. When a system’s observed behavior changes and the input was semantically the same, the change can have originated in three places: in the phrasing, in the model, or in the method used to score the output. In a results table those three are indistinguishable. A system that carries consequences has to be able to tell them apart, which means the scoring method is a component under test, versioned and reported like any other component.

FENORI VIEWThe useful part is that both readings point the same way for anyone who has to depend on the thing. If the movement is in the models, evaluation has to include perturbation with semantically equivalent inputs. If the movement is in the scoring, evaluation has to include the scorer. Neither reading permits taking a published benchmark number at face value as a statement about a deployed system.

Where the standards are heading

SOURCENIST has published The TEVV-Athlon Framework for Evaluating AI Systems as NIST AI 200-2, an initial public draft, in 2026, with public comment open until 6 October 2026. NIST describes it as a four stage method for organizations to build customized assessments against their own test, evaluation, verification and validation objectives, moving from articulate and organize, to define and construct, to apply and measure, to synthesize and interrogate. NIST states that the framework is explicitly extensible across statistical machine learning models, large language models, multi-modal models and agentic systems.[7]

It is worth being exact about the status of that document. It is a draft, it is open for comment, it has not been finalized, and nothing in it is a requirement on anyone today.

FENORI VIEWWhat we read into it is our reading and not NIST’s position. The shape of a customized, objective driven assessment is a move away from the shared leaderboard, where one number describes a model for everybody, and toward an assessment an organization constructs for itself. An assessment of that kind cannot be bought, borrowed, or cited from a vendor page, because the objectives in it are the organization’s own, and so are the entities, the obligations and the cost of getting it wrong.

The state of the field, as context

SOURCEAs context rather than as the technical basis of the argument: the Stanford Institute for Human-Centered AI reports, in the Technical Performance chapter of the AI Index Report 2025, that complex reasoning remains a challenge. Models excel at competition mathematics yet still struggle on complex reasoning benchmarks such as PlanBench, and often fail to reliably solve logic tasks even when provably correct solutions exist, which the report notes limits their usefulness in high-stakes settings where precision is critical. The same chapter separately reports that language model agents outperformed humans on programming tasks under short time budgets.[8]

FENORI VIEWWe quote both halves on purpose. They are not in tension, and a field in which both are true at once does not have a single number that describes it. Any sentence of the form “AI can now do this” is an answer to a question about a task, a time budget and a scoring method, with all three left out.

Fenori’s interpretation

Everything in this section is ours. None of it is a finding of NIST, IBM Research, Princeton, Stanford, or any of the authors cited above, and none of them has reviewed it.

FENORI VIEWStart with the version problem, because it is the smallest and clearest instance of the whole argument. ITBench exists in an earlier release with 94 scenarios and in the ICML 2025 camera ready with 102. Between them the reported FinOps result moved from 0 percent to 25.8 percent excluding anomaly detection, and the SRE result moved from 13.8 percent to 11.4 percent. The scenario set changed as well, so the pair is not a before and after of one measurement. That is exactly the point. Each number is correct about its own version and neither is comparable to the other, so a citation that names a benchmark without naming the version is not a citation. It is a number with an origin story attached to it.[1]

FENORI VIEWThe same discipline applies to any claim about a working system. A statement that a system performs at some percentage is not yet a claim, because it does not say on what, scored by which method, over how many runs, in which environment, with which permissions, at which version, across what period. Attach those and the number stops being portable. That is the honest outcome, not a failure of the exercise. Dependability was never portable in the first place.

FENORI VIEWWhat the evidence above implies for evaluating a consequential system follows from the sources rather than from taste. Repeated execution, because reliability and accuracy improve at different rates.[3] Semantically equivalent inputs, because surface changes move recorded results.[4][5] The scoring method treated as part of the system under test, because some of the observed movement may belong to it.[6] The environment as it actually is, because fragmentation, access boundaries and cross-functional handoffs are where the difficulty concentrates.[1][2] And the outcome over a period, because the claim an organization actually needs is whether the work came out right, repeatedly, in the place it was done.

FENORI VIEWNone of this makes a system correct. It makes claims about a system checkable and it makes failures locatable, which is a weaker property and a more useful one. It also costs more than quoting a benchmark, and it produces a result that only one organization can use.

Capability transfers, evidence does not

A model’s capability is a property of the model. It moves with the weights, it can be reported once, and it stays roughly true wherever that model is run. Evidence that a system can be depended on is a property of that system in that place, and it decays. It has to be produced by whoever intends to rely on it, and produced again when the model changes, the data changes, the permissions change, or the work itself changes.

The two are routinely reported as though they were the same number. They are not the same number, and only the second one answers the question an organization is actually asking.