Source: whatmodel.sh, an open source AI leaderboard I built (and launched here yesterday).
Every point comes from published benchmark results: 22,113 of them across 987 benchmarks, from 1,011 sources, each one linked to where it was published.
This chart uses data for the text models: 18,755 results for 1,211 models on 617 benchmarks.
- 14,436 results are independent evaluations: papers whose authors ran the models themselves (mostly CC BY papers from the ACL Anthology and arXiv), Epoch AI, the LMArena dataset, EvalPlus, the EleutherAI harness and others.
- 7,677 are scores labs report for their own models, labelled as lab-reported.
Everything is open: the dataset (CC BY-SA 4.0) and the scoring code (MIT) are at https://github.com/jakemanger/whatmodel, with CSV downloads at https://whatmodel.sh/data/manifest.json. Every number links back to its source, and each source's licence is recorded.
Interactive version (click any model to see its scores and sources): https://whatmodel.sh/race
Please star on github if you find it useful!
This is from an open-source AI leaderboard I built. Every point comes from a published benchmark result (22,113 across 987 benchmarks), each one linked to its source.
This chart uses the text-model results: 18,755 results for 1,211 models. 14,436 are independent evaluations (papers, Epoch AI, LMArena, EvalPlus) and 7,677 are lab-reported scores, labelled as such.
Dataset (CC BY-SA 4.0) and scoring code (MIT)
Interactive version, click any model to see its sources: whatmodel.sh/race
This is from an open-source AI leaderboard I built. Every point comes from a published benchmark result (22,113 across 987 benchmarks), each one linked to its source.
This chart uses the text-model results: 18,755 results for 1,211 models. 14,436 are independent evaluations (papers, Epoch AI, LMArena, EvalPlus) and 7,677 are lab-reported scores, labelled as such.
Dataset (CC BY-SA 4.0) and scoring code (MIT)
Interactive version, click any model to see its sources:
Jake Manger1 min read