Scoreboard / Method
How we keep score
We back open AI and we say so. The scoreboard only means something if the rules are written down and closed can win under them.
The categories
One point per category, to whichever side leads on the stated measure. A category changes hands when the measure does, not when we feel like it.
| Category | What we measure | Leader now |
|---|---|---|
| Capability | Months the best open-weight model trails the best closed model. Closed leads while every published estimate is above zero. | Closed |
| Price | Price per token for comparable work. Open leads while the typical open token is cheaper. | Open |
| Control | Whether the user can hold and run the model. This one is structural, so it only changes if the definitions do. | Open |
| Usage | Share of tokens on OpenRouter, where developers can switch models in a line of code. Open leads above 50%. | Open |
| Spend | Share of dollars spent on OpenRouter. Closed leads above 50%. | Closed |
| Recall | Whether a flawed model can be fixed after release. Structural, like control. | Closed |
| Washington | Where U.S. policy is heading. No point awarded until something becomes law. | No point |
The score over time
The tally has changed once this year.
Closed led on capability, usage, spend and recall. Open led on price and control.
Usage flipped. Open-weight models passed half of all OpenRouter tokens for good in April.
Open-weight models first crossed half of OpenRouter tokens briefly in February and stayed above it from April. Source: OpenRouter.
What counts as open
A model is open if anyone can download its weights and run it, even under a licence with conditions. A model is closed if you can only reach it through its maker's app or API. This is the line most researchers use. Our model grades show how much the licences differ inside the open camp.
How plays are tagged
- Point · Open when a move makes open models more available, more capable or better protected in law.
- Point · Closed when a move restricts open models, widens the closed lead, or strengthens the case against open weights.
- Split when one move does both, or when each side can fairly claim it.
Plays don't add up to the score. They explain why the categories move.
Sources and corrections
Every number links to where it came from, and the full source list is public. We prefer the original document to a news story about it. If we got something wrong, tell us and we'll fix it and say that we did.
Who pays for this
Open vs Closed is independent. No AI company funds it.
Use the data
Everything on this site comes from one file. Take it: data.json. There is also a feed of plays and an embeddable scoreboard.