AI Keeps Getting Cheaper — How?

In March 2023, GPT-4 launched at $30 per million input tokens and $60 for output. Today, models that match or surpass it charge a few cents. In three years, the price of equivalent capability has fallen over 99%. The key phrase: equivalent capability. The latest flagships' sticker prices haven't dropped much — some have risen. What's collapsing is the cost of a fixed level of intelligence. Some call this LLMflation: roughly a 10x decline per year.

Four Sources of Savings

**Where do the savings come from? Four sources: ① Hardware keeps iterating. Each GPU generation delivers more compute per dollar. Fundamental, but far from the biggest layer.

② Architecture is being revolutionized. MoE (Mixture of Experts): a trillion-parameter model activates only a few "experts" per token — like a 10,000-person company assigning just the most relevant few dozen people to each job. Add distillation: small models learning big models' skills at a tenth the size.

③ Inference engineering squeezes every drop of compute. Quantization, KV caching, batching, speculative decoding — individually mundane, together they cut unit costs by another order of magnitude. The 10x-cheaper "cache-hit pricing" many providers offer comes straight from these techniques.

④ Competition is smashing through the floor. Once an open-source, self-hostable model reaches comparable performance, no one holds pricing power. Add price wars, and token prices get pushed toward the cost line.

Why Flagships Stay Expensive

Then why don't flagships get cheaper? Pricing follows capability tiers: the strongest model is always scarce and commands a premium. What plunges is yesterday's flagship — just like phones: the top iPhone holds its price, but a three-year-old flagship's performance now lives in a $150 budget phone.

The Practical Takeaway

So don't chase the newest and strongest. Ask instead: last year's top-tier intelligence, now 90% off — good enough? Usually, yes.