Tokens Are the Cheapest Part

Being able to use AI is not the same as being efficient with it, and efficiency is not a low token count. It is what an accepted result costs: the tokens, the time spent checking, the rework, and the errors that get through.

One minute of an engineer’s time, at a loaded $100 an hour, costs as much as about 830,000 input tokens on Claude Sonnet 5.5, or 170,000 output tokens. Most advice on using AI efficiently is about the second number. Almost all of the cost is in the first.

Take two engineers fixing the same bug, a webhook handler that sometimes processes an event twice. Both use Claude Sonnet 5.5.

The first keeps it lean. They paste the handler into a chat, ask for a fix and take the first diff. It looks right, and reviewing it takes 25 minutes, because nothing shows whether it works. The fix misses the retry path, so there is a second round and ten more minutes. Then the change reaches staging, where a test nobody ran fails, and that costs another 45.

The second spends more. They have the agent write a test that reproduces the double processing and fails, then fix the handler and run the suite until it passes, and finish with a short summary: the failing test, the change, the passing run. The agent reads files, runs tests and tries again, and spends about fifteen times as much on tokens. Review takes ten minutes, because the evidence is already there.

Put both on one bill, at the same $100 an hour:

PYTHON

ENGINEER_PER_HOUR = 100            # loaded cost of an engineer hour, a stated assumption
IN, OUT = 2.00, 10.00              # Claude Sonnet 5.5, $ per million tokens

def cost(input_tokens, output_tokens, human_minutes):
    tokens = input_tokens / 1e6 * IN + output_tokens / 1e6 * OUT
    people = human_minutes / 60 * ENGINEER_PER_HOUR
    return tokens, people

runs = {
    "A: short prompt, first diff": cost(60_000, 6_000, 25 + 10 + 45),
    "B: failing test, agent runs the suite": cost(1_200_000, 25_000, 10),
}
for name, (tokens, people) in runs.items():
    print(f"{name:38} tokens ${tokens:5.2f}   people ${people:6.2f}   total ${tokens + people:6.2f}")

TEXT

A: short prompt, first diff            tokens $ 0.18   people $133.33   total $133.51
B: failing test, agent runs the suite  tokens $ 2.65   people $ 16.67   total $ 19.32

The first engineer spent 18 cents on tokens and $134 in total. The second spent $2.65 on tokens and $19 in total. The cheap run produced the expensive fix.

Using AI is a baseline skill now. Being efficient with it is a different skill, and the token count is a poor way to measure it.

Efficiency is what an accepted result costs #

Count everything an accepted result costs, not only the line on the API invoice:

Cost line	What drives it	What reduces it
Tokens	Context size, iterations, model choice	Lean context, caching, routing easy work to a smaller model
Review	How hard it is to tell whether the result is right	Evidence: a reproduction, a test run, the source of each claim
Rework	Results that come back because “done” was never defined	A check the work has to pass before anyone looks at it
Escaped errors	What review missed and production found	Tests in CI, staged rollout, a regression case for every miss

The lines are not the same size. At $1.67 a minute, a minute of review saved is worth more than any amount of prompt trimming for one engineer on one task, and the habits that save review minutes usually cost tokens.

Spend tokens on evidence #

The second engineer’s extra tokens went into three things: reading the files the fix depended on, running the tests, and trying again until they passed. All three produce evidence, and evidence is what makes review fast. A reviewer who sees a test that failed before the change and passes after it is checking a claim. A reviewer who sees only a diff has to work the fix out again.

The pattern holds beyond code. In Anthropic’s write-up of its multi-agent research system, token usage by itself explained 80% of the variance in performance on the BrowseComp evaluation: tokens spent searching, reading and checking bought better answers. The gate bug in an earlier post was settled the same way, by running it rather than arguing about it.

So make proof part of the task: the reproduction, the test run, the query result, the source each answer came from. Then read the proof first and the diff second.

Cut the tokens that buy nothing #

The instinct to keep token counts low is right about one thing. Most token waste is real, and some of it costs quality as well as money. Chroma’s Context Rot study tested 18 models and found that performance degrades as input length grows, even on simple tasks. Anthropic’s guidance on context engineering treats context as “a finite resource with diminishing marginal returns”, and aims for “the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome”.

The tokens worth cutting add length without adding signal:

Whole repositories, logs and documents pasted in when three files or fifty lines would do.
Long sessions that carry every earlier attempt and every raw tool result. Start fresh once a subtask is done, or clear old tool output, which Anthropic’s guidance calls one of the safest forms of compaction.
Retries with nothing new in them. A run that failed with this context will likely fail the same way again. Change the input, or stop.
Output nobody reads: long explanations, restated code, summaries of summaries.
The largest model on a task a small one handles.

A lean context is cheaper and usually better. A starved one, with the failing test or the relevant file left out to save tokens, is cheaper and worse, and the difference comes back as review time.

At volume, tokens become the bill #

Everything so far is one engineer and one task, where people cost more than tokens. A production system turns that around. The same prompt runs a million times a month, and every wasted token is multiplied by the traffic.

Take a support agent whose every request starts with the same 6,000 tokens of system prompt and tool definitions, at a million requests a month on Sonnet 5.5. At the $2 input price, that prefix alone costs $12,000 a month. With prompt caching, a hit on Sonnet 5.5 costs $0.10 per million tokens, and each hit refreshes the cache’s five-minute lifetime at no extra charge. Steady traffic keeps it warm, so the same prefix costs about $600 a month plus the occasional write when it goes cold. Real hit rates fall short of 100%, so read the cache counters rather than trusting the arithmetic.

The levers at volume:

Caching. Put the stable part of the prompt first and keep it identical byte for byte. If user content falls inside the cached span, read Prompt Caches Leak Across Tenants before you turn it on.
Routing. Claude Haiku 5.5 costs $0.10 per million input tokens on prompts up to 100,000 tokens, a twentieth of Sonnet 5.5. Send it the cases a golden set shows it handles.
Batch. Work that can wait, such as nightly evaluations and backfills, runs through the Batch API at half price, and the discount stacks with caching.
Retries. Every retry repeats the whole request. Make side effects safe to repeat with idempotency keys, then cap the retries.

Even at volume, measure cost per successful task, not per request. A cheaper model that fails one case in ten and hands it to a person can cost more than the model it replaced.

Measure the result, watch the tokens #

An efficient AI engineer tracks the expensive lines first:

Accepted results per hour of attention.
Review minutes per change.
Rework: how many results come back, and how soon.
Escaped errors: what review missed and production found.
Tokens per accepted result, as a trend.

Tokens belong on that list as a diagnostic. A run that suddenly uses five times its usual tokens is usually stuck in a loop or reading something it should not. As a target, a token budget does damage, which is Goodhart’s law at work: people under a budget leave out the failing test, the related file and the verification run, and the review time they push onto someone else never appears on the invoice.