Intelligence is not understanding.

Capability has outrun dependability

Machine capability has improved quickly and visibly. Systems now write, plan, call tools, hold long chains of intermediate work together and recover from some of their own mistakes. The improvement is real, and the benchmarks that track it are honest about what they track: whether a system can succeed at a task.

Whether an organization can depend on it is a different measurement, and the field has started taking it separately. Princeton’s Towards a Science of AI Agent Reliability decomposes reliability into consistency, robustness, predictability and safety across twelve metrics, evaluating fourteen frontier models across more than five hundred benchmark runs. Its central finding is not that agents are weak. It is that recent capability gains have not produced matching reliability gains, and that a more accurate agent can be less consistent across repeated runs than a less accurate one.

Some of that inconsistency has nothing to do with the difficulty of the task. SCORE, presented at the NAACL 2025 Industry Track, held the underlying question fixed and varied only its surface. Accuracy moved by up to ten percentage points on MMLU-Pro under semantically equivalent paraphrases, and by up to 6.1 points on AGIEval when the answer choices were reordered. The question was the same question. The world it referred to was the same world. Only the presentation changed.

The pattern repeats when the work is drawn from a workplace rather than an exam. EnterpriseBench (EMNLP 2025) builds five hundred tasks inside a simulated company, deliberately reproducing the conditions that make enterprise work difficult: data fragmented across sources that do not agree, access-control hierarchies that determine who may see what, and workflows that cross functions. The most capable models completed 41.8 percent of the tasks.

It is tempting to read results like these as model deficiencies awaiting a stronger model. Read them again and a different shape appears. Very little of what fails is inference. What fails is the system’s account of the situation it is inferring about: which record refers to which thing, what was true at the moment in question, what it was allowed to look at, what counted as established. Intelligence is the ability to reason well. Understanding is having something worth reasoning about. They are not the same property, and only one of them is currently being optimized.

What a representation actually is

A representation is not a database schema, though it usually shows up as one. It is the set of commitments a system makes about the world before it does anything. Those commitments get made somewhere: in a schema, in an integration, in a query, in a retrieval index, in a prompt, or by nobody, which is also a commitment. There are roughly eight of them, and every one is a decision a person has to make.

Take them in order. Entities decide what counts as a thing at all: whether a lease is one object or three, whether a shipment and its container are the same entity, whether an incident is an event or a state. Relationships decide what depends on what, and with what strength, because most operational failures are relationship failures rather than data failures. Identity decides when two records refer to the same thing, which is where most integrations quietly break and where the same customer becomes two customers with different risk.

State decides what is true right now, and how the system finds out when that changes. Time decides what was true when it mattered, which is a separate question and usually the harder one. Provenance decides where a claim came from and how much that origin is worth. Evidence decides what establishes a fact as settled, and what to do when two sources disagree. Authority decides who may act on any of it, and what a system is permitted to conclude on its own.

None of these are model problems. They are all specification problems, and they are all made before the first inference runs.

Time deserves particular attention, because it is the determination most often demoted to metadata. Google Research and DeepMind built the Test of Time benchmark (ICLR 2025) from synthetic data specifically so that temporal reasoning could be measured apart from memorized knowledge, splitting it into two sub-benchmarks, ToT-Semantic and ToT-Arithmetic. Accuracy varied sharply with the structure of the temporal graph, the order the facts arrived in, and the type of question asked. In one reported example, GPT-4 scored 40.25 percent on complete graphs against 92.00 percent on AWE graphs. The reasoning was constant. The shape of the represented world was not.

Authority deserves the same attention for a different reason. Microsoft Research’s CI-Work benchmark (ACL 2026 Industry) simulates enterprise workflows across five information-flow directions and measures whether a system respects the contextual boundaries that govern what may move where. Privacy violation rates ran from 15.8 percent to 50.9 percent, with leakage reaching 26.7 percent. The finding worth sitting with is the correlation: higher task utility often came with more violations. A system that does not represent who is allowed to know what will be more useful and less safe at the same time, and the authors argue accordingly for context-centric architecture rather than model-centric scaling.

Better reasoning raises the stakes

The intuitive expectation is that a stronger reasoner compensates for a weaker representation. Give the model more context, a longer window, better retrieval, and it will work around the gaps. Sometimes it does. Structurally, though, the relationship runs the other way.

A more capable reasoner acting on a wrong representation produces more confident wrong answers, faster.

Consider what capability actually buys. It buys reach: the ability to carry a premise further, connect it to more material, and express the conclusion more fluently. Every one of those is an amplifier. If the premise is that two records describe two customers when they describe one, a weak reasoner produces a shallow error that a reviewer catches in the first paragraph. A strong reasoner produces a coherent, well-cited, internally consistent account of a customer who does not exist. The second is harder to detect, harder to challenge, and arrives with more of the surface features that people use to judge whether something is trustworthy.

Capability also buys autonomy, which shortens the distance between a conclusion and a consequence. A system that only drafts gives its representation errors a human checkpoint. A system that acts converts them directly into events in the world: a payment released, a case closed, a work order raised, a record disclosed. The representation defect is unchanged. The blast radius is not.

This is why reliability research keeps finding what it finds. A benchmark that varies phrasing, ordering or graph structure is not testing whether the model can think. It is testing how much of the model’s answer was resting on the shape of the input rather than on a stable account of the subject. When accuracy moves by ten points because a sentence was rewritten, the system did not have an account of the subject. It had a response to a string.

Representation, then logic, then reasoning, then action

If the argument holds, it implies an order of construction, and the order is not the one most projects follow. Reality is what is actually the case, and no system has direct access to it. Representation is the system’s account of reality: the eight determinations above, made explicitly. Logic is what must always be true regardless of what any model concludes: policy, constraints, thresholds, the conditions under which a human decides instead. Reasoning is inference over that representation, inside those constraints. Action is the consequence, and the record it leaves.

Built in that order, each layer is testable against the one below it. Built in the reverse order, which is what happens when a capability is chosen first and the data model is discovered afterwards, none of them are. The system can only be evaluated end to end, on outputs, by people who cannot see which layer produced the fault.

In practice, taking representation first means unglamorous work. It means deciding entity boundaries before writing retrieval. It means resolving identity as a first-class concern rather than a join. It means treating valid time and transaction time as structure, so that a question about what was known at the moment of a decision has an answer. It means recording provenance on every claim, including claims a model produced, so that a conclusion can be traced to its inputs. It means expressing evidence standards and authority conditions as something executable, so that missing evidence produces escalation rather than a confident completion.

It also means evaluating the system rather than the model. NIST’s draft TEVV-Athlon framework (AI 200-2, initial public draft, open for comment until 6 October 2026) is built on that premise: a four-stage method for organizations to construct their own evaluations against their own objectives, spanning statistical machine learning, language models, multi-modal systems and agentic systems. An organization cannot borrow this. The objectives are its own, and so are the entities, the obligations and the consequences. That is a representation question before it is an evaluation question.

An honest version of this position has to acknowledge its limit. Representation work does not make a system correct. A perfect account of the world can still be reasoned over badly, and a good representation can be built on data that is simply wrong. What it does is make failure locatable. When the layers are distinct, a wrong answer can be attributed: the entity was misresolved, the state was stale, the policy was misapplied, the inference overreached. When they are not, every failure looks the same, and the only available remedy is a larger model.

The part that does not get easier

Reasoning is being commoditized. Representation is not, because it is specific to the organization that holds it. No general model can determine what an obligation means inside a particular institution, which of two records is the customer of record, what evidence that institution accepts, or who is permitted to decide. Those answers are the organization, expressed in software.

A system that has them can be improved by a better reasoner. A system that lacks them is improved into something worse: quicker, more articulate, more persuasive, and wrong in ways that take longer to find. Intelligence is the ability to reason. Understanding is having a defensible account of what is being reasoned about. Only one of them can be bought.