The cheapest model can be the most expensive worker. Procurement comparisons built around token price divide cost by the wrong thing: model usage rather than accepted work. Once review, retries, elapsed time and failed actions enter the calculation, the model is only one component in the unit economics. The commercially meaningful purchase is a configured agent system, and its denominator should be an outcome the business can accept.
The AI Daily Brief reports that Gemini 4 Argon was announced but not released. It also reports that a maximum-settings test priced Sonnet 5.5 above Opus 5.5 per task because Sonnet consumed significantly more tokens than its predecessor at the same per-token price. These are attributed reports, not independent verification of either model. Their value for buyers lies in the tension they expose. An impressive model claim can arrive without a usable product, while an attractive unit price can coexist with unattractive task economics.
Standardised benchmarks still deserve a place in procurement. SWE-bench gives a language model a codebase and an issue, then asks it to generate a patch that resolves the problem. Terminal-Bench combines a task dataset with an execution harness that connects a language model to a terminal sandbox. That structure can help a buyer screen candidates, identify promising configurations and track changes under repeatable conditions. It also demonstrates why the resulting score is not a verdict on the model alone. The task, harness, available tools and acceptance mechanism all help shape the result.
The distinction is visible in products already offered to developers. Google lists file operations, shell commands, web fetching and custom integrations among the features of Gemini CLI, while OpenAI describes Codex CLI as a coding agent that runs locally on the user’s computer. Those descriptions place the model inside different operating arrangements. The harness decides what context can enter, which tools can act and where approval interrupts execution. Buying by model name alone ignores machinery that can determine whether intelligence becomes useful work or expensive supervision.
Pricing accepted work begins as an internal management calculation, not a demand that every supplier accept a success fee. Define acceptance before running the test. For a coding workflow, an accepted outcome might be a patch that passes specified tests and receives the required review approval. For an accounts workflow, it might be a correctly processed exception with an auditable record and no unauthorised action. Calculate total cost across supplier charges, infrastructure, review time, reruns and remediation, then divide it by accepted outcomes. That denominator puts a cheap but labour-intensive system on the same commercial footing as a dearer system requiring less intervention.
Consider an explicit hypothetical in which Agent A completes more assigned tasks than Agent B. Agent A also creates ambiguous failures that require lengthy investigation, while Agent B stops early and identifies what it cannot resolve. Agent B could produce accepted work at lower total cost where reviewer time is scarce and undetected errors carry material consequences. If the supervision burden reversed, the commercial choice could reverse too. The useful question is therefore not which agent completes more work, but which one completes acceptable work inside the relevant risk limit.
A credible evaluation should draw a narrow task set from the workflow being funded. Give each candidate the same starting material, authority and definition of acceptance. Set permissions to match the proposed deployment rather than granting broad access to improve a demonstration. Record accepted outcomes, human interventions, end-to-end latency and total cost. Count abandoned attempts and reruns rather than allowing them to disappear from the average. Read the failure logs to see whether the system asks for help at a sensible point or continues beyond its competence. Repeat the exercise after material changes to the model, harness or integrations.
Pricing accepted outcomes also has a contractual meaning, although it need not mean paying only for success. A buyer can retain metered pricing while negotiating reporting obligations, performance thresholds or service credits tied to the agreed task set. A pure success fee may encourage arguments about acceptance or push a supplier towards easier work. The contract should instead make the economics inspectable. Buyers should retain the logs needed to explain performance changes and the right to test model substitutions under deployed permissions and integrations. A supplier unable to reproduce its result in that setting has supplied screening evidence, not purchasing evidence.
If frontier model quality converges while harness quality remains uneven, advantage will conditionally shift towards suppliers that make reliable work easier to configure, observe and govern. That prediction depends on convergence, but the procurement discipline does not. Benchmarks can decide who earns a trial. Accepted outcomes should decide who earns the contract.
sources
Prepared with assistance from a Useful Engine agent workflow. Factual and editorial checks are part of the publishing process.