I cut 28% of my coding agent's input tokens. The cost fell about 4%.
I replaced the 21,769-character instruction template that ships with the Codex CLI with a file of 899 bytes. Across 24 real coding runs, total input tokens fell by 28.03%. At published rates the estimated cost fell by 4.31%, and the trimmed version passed fewer tasks.
This is a negative result. I am publishing it because the first of those numbers is the one that gets quoted.
Codex CLI 0.159.2, gpt-6.1-sol, low reasoning. Cost is an estimate at published rates, not a bill.
Why I expected it to work
The template goes out with every request. A default request body was 57,049 bytes. With the short file it was 35,942 bytes, 37.0% smaller. Fewer tokens on every turn of every task looked like an easy saving.
Setup
- Codex CLI 0.159.2, model gpt-6.1-sol, low reasoning effort. Tools, permissions and everything else were the same in both arms.
- Six kinds of small code-repair task: merging intervals, LRU capacity, a backoff ceiling, topological sort, JSON merge patch, and an output cursor.
- Each task ran twice per arm, 24 runs in all, one after another, each in a fresh directory.
- The change used a supported option,
model_instructions_file. The short file kept the rules on task scope, permissions, protecting secrets, finishing the work, testing, and reporting honestly. The tool definitions, about 20,462 bytes, stayed. - The protocol and the pass thresholds were written down and frozen in a commit before the run. It ran once, with no retries and no extra diagnostic calls.
That last point is why I trust the result more than I like it.
Results
| Sum over 12 tasks | Default | Trimmed | Change |
|---|---|---|---|
| Input tokens, including cached | 463,258 | 333,427 | −28.03% |
| Cached input | 274,688 | 152,192 | −44.60% |
| Uncached input | 188,570 | 181,235 | −3.89% |
| Output tokens, including reasoning | 4,465 | 5,220 | +16.91% |
| Estimated credits at published rates | 11.23 | 10.75 | −4.31% |
| Tasks passing the frozen checks | 11 of 12 | 9 of 12 |
Total input fell in every one of the 12 pairs, by between about 8.8% and 32.4%. Estimated cost fell in six pairs and rose in the other six.
Where the saving went
The rates I used, looked up on October 1, 2026 for the Standard tier: 50 credits per million uncached input tokens, 2.5 per million cached input tokens, and 250 per million output tokens.
- Cached input: 122,496 fewer tokens, worth 0.31 credits. Cached tokens cost a twentieth of uncached ones, and most of the instructions I removed were already being served from cache.
- Uncached input: 7,335 fewer tokens, worth 0.37 credits.
- Output: 755 more tokens, costing 0.19 credits. Output is priced at five times uncached input.
Net: 0.48 credits saved out of 11.23.
The quality gate failed
The default arm passed 11 of 12 tasks and the trimmed arm passed 9 of 12. That missed the threshold I had set in advance.
When I went through the four failures afterwards, the code itself was fine in all of them: every one of the 24 submitted source files passed the functional checks offline. Three failures were my validator rejecting a legitimate call, int.bit_length(), that was not on its allow-list. The fourth was a test command with an environment-variable prefix that the checker did not recognise.
I kept the original verdict anyway. The thresholds were fixed before the run and the run missed them. The later review is recorded separately and labelled post hoc, and it does not change the recorded outcome.
How I would have misread it
If I had looked only at total input tokens, I would have reported a 28% saving. That number hides three things:
- Which tokens went away. The cheap ones did.
- What happened to output, which is the expensive side.
- Whether the work still got done.
Limits
- 24 runs, six small tasks, one model, one CLI version, one reasoning setting.
- The cost is an estimate from published rates. I have no invoice for these runs, and the account balance did not move in a way I could attribute to them.
- The short file dropped some detailed workflow instructions, so this was not a like-for-like compression of the same guidance.
- Cache state was not controlled.
- Wall time was 227.65 seconds for the default arm and 257.81 seconds for the trimmed one in this sample. I draw no conclusion from that.
What comes next
The next version of this experiment would fix the validator first, use real repositories and longer tasks, control and record whether the cache is warm, and then judge quality, uncached input, output and total cost together. The idea worth testing is a short task-specific instruction file behind a stable shared prefix, so the prefix keeps hitting the cache.
The experiment lives in a private repository for now. The protocol, the frozen commit and the raw run archive exist, and I have not published them yet.
Sources
- My own experiment record: the protocol, the run archive and the cost report for this work package.
- Codex pricing, for the rates.