LLM Benchmarks - Directory w/ AI Reviews
Choosing the right LLM for a task requires rigorous comparison across dimensions like reasoning, coding, multilingual ability, and cost. LMSYS Chatbot Arena uses crowdsourced human preference ratings to rank models on open-ended tasks. HELM provides standardized benchmark suites for academic and industry comparison, while the Hugging Face Open LLM Leaderboard tracks open-source model performance. Artificial Analysis adds infrastructure metrics like throughput and latency to the evaluation picture.
1
4.9
New
HELM
Free
Free Plan
Open Source
HELM by Stanford evaluates language models across dozens of scenarios measuring accuracy, robustness, and fairness.
2
4.9
New
LMSYS Chatbot Arena
Free
Free Plan
Open Source
LMSYS Chatbot Arena is a crowdsourced LLM evaluation platform developed by LMSYS Org that ranks language models through blind, randomized head-to-head battles judged by human users. Visitors submit prompts and vote on anonymous model outputs, with results aggregated into an Elo-style leaderboard tha
3
4.8
New
Hugging Face Open LLM Leaderboard
Free
Free Plan
API
Open Source
The Open LLM Leaderboard by Hugging Face is a comprehensive benchmark tracking platform that evaluates open-source language models across standardized academic benchmarks. The leaderboard automatically runs models through evaluation suites including MMLU, ARC, HellaSwag, TruthfulQA, Winogrande, and
4
4.7
New
Artificial Analysis
Free
Free Plan
Artificial Analysis independently evaluates AI models across quality, speed, price, and throughput, comparing API providers for developers.
5
4.4
New
Evalverse
Free
Free Plan
Open Source
Evalverse is an open-source unified LLM evaluation framework integrating multiple benchmark suites in one interface.