Benchmarks Explained
Short answer: above 70% is frontier-class in 2026 — the model can resolve most real GitHub issues on its own. The current leader, Claude Fable 5, sits at 93.4%. But "good" depends entirely on the year you're asking in — here is how to read the number.
| Score | Tier | What it means in practice | Models today |
|---|---|---|---|
| 90%+ | Absolute frontier | Resolves nearly all real GitHub issues autonomously, including multi-file changes. Only a handful of models have ever crossed this line. | 2 |
| 70–90% | Frontier-class | Handles most real-world engineering tasks end-to-end. Anything in this band is a serious daily-driver coding model in 2026. | 5 |
| 50–70% | Capable mid-tier | Solves the majority of well-scoped issues but needs human review on complex, multi-file work. Often the best value for volume tasks. | 2 |
| Below 50% | Budget / legacy | Fine for boilerplate, autocomplete, and simple fixes — but by 2026 standards this is not a model to hand a real issue to unsupervised. | 8 |
SWE-bench is the fastest-moving major benchmark. In early 2024 the best agents resolved under 20% of issues. Late 2024 pushed state of the art to roughly 50%, mid-2025 crossed 70%, and today's leaders are above 90%. That means a "good" score has roughly doubled every year — a model praised in a year-old article may be budget-tier now. Our SWE-bench leaderboard always reflects the current published scores.
One nuance worth knowing: most headline numbers refer to SWE-bench Verified, not the full benchmark — a human-validated subset that removed broken and ambiguous tasks. Scores on the two are not interchangeable.
And past ~70%, the score alone should stop deciding for you. Price per token and context window determine what a model costs to actually run on your codebase — that is why the leaderboard shows both, and why the best AI for coding is not automatically the top-scoring one.
Anything above 70% is frontier-class — the model can resolve most real GitHub issues autonomously. The current leader, Claude Fable 5, scores 93.4%, and 7 of the 17 models we track clear the 70% bar. The median tracked model sits at 54%.
The bar has moved brutally fast. In early 2024 the best agents resolved under 20% of SWE-bench issues. By late 2024 roughly 50% was state of the art, mid-2025 pushed past 70%, and today the leaders are above 90%. A score that led the field two years ago is budget-tier now — always check the date on any score you read.
No. Above roughly 70%, price and context window usually matter more than a few extra points. A model a few points behind the leader at a fraction of the cost is the better pick for everyday work — reserve the outright leader for the hardest tasks.
Mostly, with one caveat: providers sometimes report scores using different agent scaffolds, tool access, or attempt counts (pass@1 vs best-of-N). The headline numbers on official leaderboards use SWE-bench Verified under comparable conditions, which is what our leaderboard tracks.
Each task is a real GitHub issue from a real open-source repository. The model gets the codebase and the issue text, and must produce a patch that passes the repository's own tests. It measures the messy, multi-file work engineers actually do — not toy puzzles.
Newsletter
We track new benchmark publications. When a model takes the top spot, you'll know first.
No spam. Useful updates only. Affiliate disclosures always clearly labeled.