Fable-5-level results from Opus 5. At half the token price.
Claude Code running Claude Opus 5 with vexp 3.1.1 resolved 81.7% of the SWE-bench Pro public set. Anthropic reports 80.3% for Claude Fable 5, whose tokens cost twice as much. Every session, patch and evaluation is published.
Three numbers, one benchmark.
Our run against the figures Anthropic published for its own models.
Single run, 731 tasks, 95% CI 78.7–84.3 · Anthropic figures vendor-reported · Every session published → GitHub
| System | Resolved | Token price, in / out per Mtok | Source |
|---|---|---|---|
| Claude Opus 5 + vexp 3.1.1 | 81.7% · 597/731 | $5 / $25 | this run, every session published |
| Claude Fable 5 | 80.3% | $10 / $50 | Anthropic, vendor-reported |
| Claude Opus 5 | 79.2% | $5 / $25 | Anthropic, vendor-reported |
Same level as the pricier tier. Half the price per token.
The interval of this run, 78.7 to 84.3, contains both published points. What the numbers establish is that an Opus 5 agent with vexp lands at Fable 5's level. The 1.4 points above it are inside single-run noise, so we say “level with”, not “beats”.
The three numbers do not come from the same scaffold. Anthropic's come from its own harness; ours from Claude Code with a 120-turn cap. That is normal for SWE-bench Pro: public numbers are routinely produced on different scaffolds, and the scaffold moves the result by tens of points.
What is not in doubt is the price. Every component of Fable 5's list price is twice Opus 5's: base input, cache writes, cache reads, output. A run at Fable-5-level resolution on Opus 5 tokens pays half for everything it replays.
An Opus 5 agent with vexp reaches Fable-5-level resolution on SWE-bench Pro while paying Opus 5 prices. Every session behind the number is public.
That this run beats Fable 5 with statistical certainty, or that vexp adds a fixed number of points to a model. The repository publishes no same-scaffold run without vexp, so it does not isolate vexp's own contribution.
Where the money goes.
Token usage of the 731 sessions at Anthropic's list prices for Claude Opus 5.
| Component | Tokens | List price | Cost |
|---|---|---|---|
| Base input | 35,689 | $5 / Mtok | $0.18 |
| Cache writes (1-hour) | 31.8M | $10 / Mtok | $318.28 |
| Cache reads | 1,054M | $0.50 / Mtok | $527.12 |
| Output | 10.4M | $25 / Mtok | $260.49 |
| Total | $1,106.07 · $1.51 / task |
Claude Code's own meter for the run is $1,111.55, $1.52 per task, within 0.5% of the list-price recomputation.
Ninety-seven percent of billed input is cache reads. An agent loop replays its context every turn, and the model's per-token price sets the bill. That is why the choice of model is a saving in itself, and why fewer tokens per turn is the other one.
Prices: platform.claude.com, read 2026-09-05. Arithmetic: COMPARISON.md.
How we ran this benchmark.
One run, the official harness, every artifact published. No cherry-picking.
Don't trust us. Run it yourself.
The scripts that ran it, the task set, the harness and every result are in one public repository.
$ git clone https://github.com/Vexp-ai/swe-bench-pro-opus-5-vexp.git
$ cd swe-bench-pro-opus-5-vexp && gh release download run-2026-09-05
$ tar xzf swebpro-vexp-3.1.1-binary.tar.gz -C harness
$ cp <your-vexp-licence> harness/bench-license.jwt
$ export CLAUDE_CODE_OAUTH_TOKEN=...
$ cd harness && ./run-instance.sh 1 # one task
$ ./orchestrate.sh 12 3 && ./eval-incr.sh # all 731, official evaluation
Requires Docker, an Anthropic account, and a vexp licence without the 2,000-node cap. The results archive on the release page holds every session, patch and harness output.
Use code BENCHMARK at vexp.dev/#pricing for 14 days of vexp Pro — free.
No credit card required.
Same model. Better context.
Half the price of the next tier.
81.7% on SWE-bench Pro with Claude Opus 5. Every session published.
All on your machine. No cloud. No account.
or npm install -g vexp-cli
One-time email with install links. No spam.
Free tier · No account · No credit card · Zero network calls