Is the Tokenpocalypse Coming? Token Prices Are Crashing — So Where Does the Inference Cost Go?

A token is the smallest unit an LLM (large language model) uses to process text and put a price on it. Cloud APIs bill input tokens and output tokens separately, in dollars per million, and output typically runs four to six times pricier than input. Lately the phrase "AI tokenpocalypse" has been going around — the worry that runaway inference cost will break the AI boom. Yet token prices are, right now, still falling. Hold performance constant and cloud inference has dropped from roughly $60 per million tokens at the end of 2021 to about 6 cents three years later.

That mismatch is the starting point. Read cost along the price axis alone and it doesn't add up. The cost of a single query is unit price × tokens consumed per query, and on top of that sits the question of whose ledger the resulting cost lands on (incidence). Price, consumption, and incidence. We take the three axes one at a time, then ask where the price stops falling and how the map splits.

Price: The Conventional Wisdom Is Only Half Right

The plunge in same-performance price is a measured fact. a16z called the trend "LLMflation" — same-performance LLM inference price falling roughly 10x a year — and Epoch AI measured it at anywhere from 9x to 900x a year depending on task difficulty. On price alone, there is no "pocalypse."

But the plunge comes with a condition: it holds only when you fix quality. The newest, top-performing tier is a different story.

The newest, top-performing tier doesn't ride this plunge curve well. The frontier flagships (Opus 4.8, GPT-5.5) run two to three times the 2024 GPT-4o ($2.50/$10), and the reasoning model o1 runs six times as much. Reasoning models do shed price as the generation descends, though: the successor o3's output price of $8 is one-seventh of o1's. So what stays expensive is not the entire "reasoning" class but the launch premium attached to the best quality at any given moment. The point: same performance plunges, but chase the best performance at all times and that premium keeps the price from coming down. And, as we'll see, the token-gorging growth workloads use exactly this premium tier.

Consumption: Even as Price Falls, Total Volume Rises

The second axis is the tokens a single query consumes. This moves in the opposite direction from price. A reasoning model generates hundreds to tens of thousands of "thinking tokens" before it answers, billed at the output rate. Even a fixed model can, by burning more of these thinking tokens and allocating test-time compute well, outperform a model 14 times its size. You buy performance with tokens. Context windows have grown to a million tokens, so a single long-form query swallows hundreds of thousands of tokens, and an agent's multi-turn tool calls push the same way. The system regime seen in What Comes After the Transformer — the Question Is Wrong directly enlarges token consumption.

There's an inverted intuition here. Thinking tokens go out whether they buy performance or not. When a reasoning model ultimately gets a hard problem wrong, the thousands or tens of thousands of thinking tokens burned to get there are billed all the same. A token is not the purchase of capability but the occurrence of consumption. What reliably rises is consumption, not capability.

Total volume is settled by measurement. Google reported processing 3,200 trillion tokens in the single month of May 2026 — about 7x year over year.

That falling prices lead to using more is not news. The Jevons paradox (1865 — the observation that when a resource becomes cheaper to use, total consumption actually rises) was likened to AI by Microsoft's CEO right after DeepSeek's release. Since this is conventional wisdom, we leave it as a mechanism footnote rather than a headline.

So wouldn't efficiency gains hold consumption down? Techniques like quantization (compression that lowers weight precision to cut computation) and small-model routing are real, and much of the price plunge comes from them. But efficiency is not a brake on consumption; it's an engine that spurs it. Get cheaper and you attempt more complex things, new workloads open up, consumption rises. That said, this is only about total consumption rising. Whether that increase pushes total dollar cost up is a separate question from volume, and we take it up right below.

What's settled is volume (token throughput), not the trajectory of total dollar spend. The number people usually find frightening is hyperscalers' 2026 capital expenditure — guidance totaling roughly $725 billion, up 77% year over year. But this money is capital expenditure (capex) that props up training and inference together, not the operating expense (opex) that goes out every month on inference serving. The same GPUs both train and infer, so you can't carve inference-serving cost out of this headline number. On the dollar trajectory of inference, the most one can say now is a qualitative observation: that each AI query is expensive enough — an order of magnitude beyond traditional search — to eat into cloud providers' per-query margin.

Incidence: Cost Doesn't Vanish, It Moves

The third axis is who bears the cost. This is the problem of cost incidence (whose ledger or silicon ends up absorbing the cost). Cost doesn't get cheaper; it changes form and moves. On-device AI shifts the provider's operating expense (opex, the bill that comes every month) onto the capital outlay of silicon the user has already bought (capex, the kit you buy once). Apple's hybrid (on-device plus non-storing cloud inference) is one such transfer. System orchestration and output verification (A Machine That Has Never Seen a Board Knows the Board — Understanding, Mimicry, or the Wrong Question?), and the liability cost of agent error (AI Agents and the Accountability Gap: the Work Is Delegated, the Respondent Is Not), go to application vendors and system integrators.

One rebuttal holds that on-device genuinely eliminates cost. Push it down to the device and the provider's opex disappears; the user runs on a chip already bought, so the per-query cost is zero. But the capex didn't vanish — it's a transfer the user prepaid in the price of the device. It's invisible, not gone (consistent with the verdict in On-Device AI: Cost and Jurisdiction, Not Chips, Draw the Line).

What's more, the deployment axis (device or cloud) and the system axis (simple regime or complex regime) are orthogonal. Run a complex regime — reinforcement learning, verification, agents — inside the device and capex actually grows. So the cost hasn't gone somewhere and vanished; it merely changes form. Under a complex regime it swells instead.

The Physical Floor: How Far Can Price Fall?

We've walked the three axes, but the first — price — still holds an unanswered question: where does this fall stop? The decline so far has come mostly from algorithmic and architectural efficiency, and this efficiency routes around the raw-material cost of memory. So even as the price of an HBM (high-bandwidth memory) stack rises generation to generation (HBM3 about $200 → HBM4 about $500), token prices can keep falling for a while. The real floor comes not from raw materials but from supply scarcity. In 2026 the actual bottleneck in AI hardware is advanced-packaging allocation, and CoWoS capacity is sold out. SK hynix has said HBM orders already exceed its production capacity for the next three years. This allocation bottleneck props up the price of GPU time and could, at some point, halt the price decline.

Here is the strong version of the earlier worry. If the price decline hits this physical limit and stops while consumption keeps rising, total cost genuinely dents margins and investment. This path is logically possible. But the current data isn't at that point yet. Price is still plunging, and multiple providers are cutting prices competitively. So we leave this as a falsifiable signal to watch — without sliding into an assertion that it "is coming" or a diagnosis that it's "a bubble." When the physical floor bends the price curve is the thing to watch.

The Map: Two Regions

Overlay the axes and a map appears. Per-query cost is unit price × consumption, and that product splits by workload. On top of it sits who the cost goes to (incidence). So the question "does the tokenpocalypse break the boom" is only half right, because it frames the problem on the wrong axis (price). The map splits into two regions.

In the stagnant region the rebuttal is right. Fixed prompts and structured classification/extraction tasks don't see per-query tokens explode, so a price plunge is a total-cost plunge. Here cost doesn't change form; it genuinely gets cheaper. The growth region is different. The tokens rising here belong to the premium tier (reasoning, frontier) whose price barely falls, and per-query consumption explodes on top. So as long as you chase the newest, best performance, the price decline doesn't come down to the cost of these queries. Still, don't blur this into "consumption outran the price decline." Same-performance price falls 10x a year and throughput rose 7x, so on the same quality basket alone, spend actually drops. The pressure in the growth region comes from "more" and "at a pricier tier" overlapping — not from consumption arithmetically beating price. So whoever says on one axis that "everything gets cheaper" or "everything blows up" is wrong both ways.

Whether the growth region's rising spend is a crisis or healthy growth is a separate question. Spend rising is not itself a crisis; it may mean usefulness has grown that much. What sorts this is not cost but unit-economics (whether the value created per query exceeds the cost per query). This piece is anchored in the cost mechanism, so it won't pronounce on the value side. People commonly cite here, as a sign of a bubble, the analysis that 2026 AI capex growth outpaces revenue growth. But in the early build-out of a capital-intensive industry, capex outrunning revenue is a normal phase that railroads, telecom, and cloud all passed through, so on its own it can't separate a bubble from healthy growth. The sustainability of the spend axis remains an open question hinging on unit-economics, and the answer hasn't been observed yet.

The honest map is this. The price table advertises a plunge, but whether that plunge carries through to total spend scatters across different axes — workload, capex, power — and won't resolve into one. What's certain is that cost hasn't vanished, it has moved, and that in growth workloads the price decline doesn't offset that cost. How much it pushes total spend up is still open.

For the people who build, this map tells you where to set your token budget. Structured tasks can simply enjoy the cloud price decline. In growth tasks the lever is consumption, not price. Keep recurring prefixes from being paid for twice with prompt caching (a cache read runs about one-tenth of the input price), route easy queries to a small model, cap output tokens, and open the thinking budget only as far as needed. Rather than passively waiting for the price to fall, which tier you run and how decides the cost of the same question.

One gap remains: when does the price decline hit the physical floor of power, HBM, and packaging (Packaging Sets the Ceiling on AI Accelerators: Why the Back End Became the Center of Gravity)? When that point arrives, the growth region's map gets redrawn. The "tokenpocalypse" takes on real meaning not when price collapses but when that fall stops at the physical floor — and the current data isn't there yet. Until then, wherever the cost moves, the question of who bears it stays — erased from the price table.