Insights - The spec is 0.04 percent. The misunderstanding is 7 million tokens.

Thursday afternoon, a review of our spec pipeline. The question on the table: use cases, acceptance criteria and non-functional requirements were built for humans. Is there a better way to write them for a model?

We looked at the numbers first. The spec text is about 0.04 percent of the tokens a clean run reads. A clean run costs 3.0 million tokens. A run that the agent reviewer blocks costs 10.6 million. Of the last 252 runs, 72 were blocked. We read those 72. Two causes, roughly half each: the story overriding the criteria, and constraints nobody wrote down.

So the spec is not the cost. The misunderstanding is. Rewriting the same prose in a new format saves nothing. Making the model get it right the first time saves 7 million tokens per blocked run.

What does the model skip? Actors. Preconditions. The main scenario. These sections tell the story of the acceptance criteria again, in prose. When the story and the criteria agree, the prose adds nothing. When they disagree, the worker follows one and the reviewer the other. That is a block. One of our specs stated the same baseline three times in three sections. That is not clarity. That is a spec that does not trust its own criteria.

The model does not need a new language. It needs a test to pass, a list of things it must not do, and a clear place to plug in.

Four things help, in this order.

A test or an example it can run. Two of the criteria in that spec were fixture pairs: this input, that output. Those two never caused a block. The prose criteria did.

A list of limits and non-goals. A study on design constraint compliance puts a number on it: fewer than half of the patches that pass their tests also satisfy the design constraints. Passing tests says almost nothing about compliance. Stating the constraints reduces the violations. Not to zero.

The seam. File path, function signature, call site. The failure mode we know best is “built, but never wired in”. A typed signature beats a scenario.

The invariants. Which existing behaviour is load-bearing. That is what stops “fixed one thing, broke the neighbour”.

The uncomfortable part: our traceability from criteria to tests points at tests the same agent wrote afterwards. The spec checks itself. The missing piece is a holdout test the spec author writes and the worker never sees. Not built yet.

One remark. We cap specs at 10 criteria and one concern each. Keep that. The curse of instructions is measured: give a model one instruction and it follows it about 90 percent of the time. Give it ten at once and the rate drops to 44 percent, on the best model in the study. A richer template with more rules is the opposite of help.

The new form ships this week, advisory first. In 30 days we compare the block rates. If they do not move, we say so here.

Use cases were built for humans. That is not their problem. Their problem is that they do not trust themselves.