Model prices move faster than most budgets. Stanford HAI’s AI Index Report 2025 documents how quickly the cost of GPT-3.5-level language performance collapsed—and how smaller models closed gaps that used to require hundreds of billions of parameters.
What the Index measured
The AI Index tracks the cost of querying a model that matches GPT-3.5’s reported MMLU score of 64.8. In November 2022 that cost was about $20 per million tokens. By October 2024, Gemini-1.5-Flash-8B delivered equivalent benchmark performance at $0.07 per million tokens—more than a 280-fold reduction in roughly eighteen months.
Depending on the task, LLM inference prices fell anywhere from 9 to 900 times per year over the period the Index analyzes. The headline is not that every workload got 280× cheaper overnight. It is that capability at a fixed quality bar became radically less expensive.
Smaller models, same threshold
In 2022, the smallest model clearing 60% on MMLU was PaLM at 540 billion parameters. By 2024, Microsoft’s Phi-3-mini reached that threshold with 3.8 billion parameters—a 142-fold reduction in model size for the same performance band.
That shift matters for product teams: many workflows no longer need the largest available model to clear a useful quality bar. Routing, caching, and evaluation matter more than chasing parameter counts.
What this changes in practice
Cheaper inference expands what you can put in production: more retrieval calls, more retries with review, more batch jobs that would have been too expensive two years ago. It also makes it easier to overspend on poorly designed workflows—volume scales with price drops.
Treat the 280× figure as a signal to re-baseline unit economics, not as a guarantee that your current architecture is efficient. Measure cost per accepted task, not cost per token alone.
Keep the comparison honest
Benchmark-matched cost is not the same as task-matched cost. MMLU equivalence does not mean two models behave the same on your support tickets, code reviews, or internal search.
Use public Index numbers to set expectations, then run your own evaluation set before you change providers or shrink models in a live workflow.