The Token Cost Crunch: Why Falling AI Prices Are Producing Rising AI Bills

The price of an AI token has collapsed, and enterprise AI bills have gone up anyway; this has become the defining budget problem of enterprise in 2026. The math is simple: a bill is the price of a token multiplied by the number of tokens consumed. The industry has spent three years watching the price fall and almost none of it watching the volume, which has grown faster than any budget model accounted for.

The FinOps Foundation's State of FinOps 2026 report, drawn from 1,192 practitioners responsible for more than $83 billion in annual cloud spend, found that 73% of organisations reported their AI costs exceeded original projections. The share of FinOps teams managing AI spend rose from 31% in 2024 to 63% in 2025 and to 98% in 2026.

The Foundation locates 80% to 90% of AI expenditure in inference rather than training, and AI cost management is now the single most sought-after skill named by practitioners, with 58% prioritising it for development over the next 12 months.

A discipline that spent a decade governing reserved instances is being asked to forecast token consumption for workloads that have existed for less than a fiscal year.

Ramprakash Ramamoorthy, Director of AI Research at Zoho Corp, said the structural reason is that inference behaves unlike every cost line that preceded it. “Inference is the part of AI that runs every time a user makes a request, so its cost scales with usage in a way training never does,” he said. He added that the timing of when the cost becomes visible is what makes it dangerous. “For many organisations, the cost only becomes visible once they move past pilots into real volume, and by then the architecture is already locked in.”

Uber spent a year's AI budget in four months, and the rollout was a success

The most instructive number in enterprise AI this year came from a company whose deployment worked exactly as intended. Uber rolled out Claude Code to its engineering organisation in December 2025; adoption climbed from 32% to 84% of roughly 5,000 engineers by March 2026, and by April, the entire 2026 AI budget was gone.

The rollout succeeded on every measure it was designed against, with Uber reporting 70% of committed code being AI-generated and 11% of backend updates written by fully autonomous agents, at monthly costs per engineer averaging $150 to $250 and power users running between $500 and $2,000.

Praveen Neppalli Naga, Chief Technology Officer at Uber, was quoted as saying that he was back to the drawing board because the budget he thought he would need had already been blown, having spent $1,200 in tokens during a single two-hour personal demo.

Engineers were using the tool for parallel agent execution, large-scale codebase refactoring, automated test generation and backend code production, which is precisely the workload it was built to handle, so the productivity result and the financial result both followed from the same correct usage.

Uber compounded the dynamic by ranking engineers on internal leaderboards according to Claude Code usage, which created a cultural incentive to consume more tokens and translated directly into faster budget burn. The teams driving adoption were not the teams managing the spend, and that organisational separation turned out to be the load-bearing flaw.

The ride-hailing company has since capped spending at $1,500 per month per tool. Andrew Macdonald, President and Chief Operating Officer at Uber, has been publicly sceptical about what the money bought - that the connection between rising Claude Code usage and consumer-facing innovation is not there yet, and that while more may implicitly be getting shipped, it is very hard to draw a line from those statistics to producing more useful features meaningfully.

Uber's total research and development spend reached $3.4 billion in 2025, which makes the overrun a story about a pricing model that enterprise finance has not learned to hold, rather than a scale story. The 2026 budget was set in 2025, before token-burning agents existed at production scale.

Budgets break because the costing arrives after the deployment

The failure is one of sequence rather than technology. Teams ship first and measure afterwards, and at chatbot scale, that order remains survivable because the volume stays small enough that the arithmetic never bites hard enough to register. At agentic scale, it stops being survivable, because the same architectural decision that cost almost nothing during the pilot is now executing thousands of times a day against a bill nobody modelled.

Avinav Nigam, Founder and CEO of TERN Group, said the distance between demo and production is where most AI business cases collapse. “The inference cost problem is the thing most AI roadmaps do not have a line for, and then suddenly it is the biggest line on the P&L,” he said.

He explained that TERN's own deployment made the point concrete, since TERN has built Maitha, which runs live two-way video interviews, generates questions in real time, processes audio and scores responses simultaneously. “When you move from a pilot to a live deployment at that level of concurrency, the compute bill is not what anyone budgeted for,” Nigam said, adding that most organisations discover this after they have already committed to a production rollout.

His prescription is arithmetic performed before commitment rather than after, which means modelling the inference cost at ten times the expected volume before choosing a production architecture and confirming the business case still holds.

Nigam explained that the failure pattern is running a pilot with fifty transactions, building a business case from the unit economics, and then discovering the cost structure breaks at five thousand. “If it does not, you need a different architecture, not a bigger budget,” he said.

The market data corroborates the sequence he described. Analysis of 2.4 billion enterprise API calls found the blended cost of AI fell 67% year on year, from $18.40 to $6.07 per million tokens between Q1 2025 and Q1 2026. The same analysis put the median blended cost at $2.31 per million tokens for organisations running a tiered model architecture.

Organisations sending every workload to a frontier model paid $18.40 per million tokens for the same work. That 87% gap is the financial consequence of a single architectural decision, usually taken early and rarely revisited, rather than any pricing outcome the market imposed.

Agentic AI multiplies the calls, and the multiplier carries the whole story

Gartner predicts that by 2030, performing inference on a model with 1 trillion parameters will cost providers more than 90% less than in 2025, driven by improved hardware and model design, inference on edge devices, and inference-specialised chips. None of that is expected to reach enterprise invoices intact.

Will Sommer, Senior Director Analyst at Gartner, told CIO Dive that cheaper tokens unlock cheap capabilities and expensive ones simultaneously. “Yes, token costs are coming down; that is going to unlock relatively low-value capabilities that will become embedded in existing ecosystems,” Sommer said. “It will also unlock higher-value applications.

Those applications are going to be more expensive, not less.” He noted that provider efficiency gains are being retained rather than passed through because a lot of the largest labs are currently losing money and need lower costs relative to revenue to change that.

The multiplier is where the economics turn. Gartner assesses that upgrading from a generative AI chatbot to an agentic assistant means every single query costs five to 30 times more tokens, which is a different claim from the assistant simply making more queries.

EY's analysis found the cost of a single agentic customer-service interaction rose from roughly $0.04 in 2023 to $1.20 in 2026, a roughly 30-fold increase driven by orchestrated multi-tool workflows replacing linear chatbot exchanges, and Goldman Sachs estimates agentic AI could push token consumption up 24-fold by 2030.

Ramamoorthy said the mechanism is context accumulating across a reasoning loop. “Agentic systems multiply inference because a single request can trigger many model calls as the agent reasons, calls tools, and revisits context,” he said. “Left unmanaged, the token count for one task balloons and so does the total cost.”

He explained that Zoho's response operates at runtime rather than in the architecture: “At Zoho, we monitor context usage in real time and prune or summarise it smartly, which keeps long-running agent tasks affordable.”

Nigam described the same multiplication with the failure cost attached. “A single agentic workflow that looks like one user action might be making fifteen to twenty LLM calls under the hood. Each one costs money. Each one adds latency,” he said.

He noted that chaining makes partial failure expensive in a way single-call systems never were, since a failure at step eight of a fifteen-step pipeline means seven inference calls have already been paid for and produce nothing.

Coinbase halved its bill without slowing anyone down

Coinbase faced an identical arithmetic and resolved it in the infrastructure rather than in policy. Brian Armstrong, Chief Executive Officer at Coinbase, disclosed in June 2026 that the company had cut its AI spend nearly in half while token usage continued to grow, and he was direct about the method.

“How to keep AI spend flat while token usage grows exponentially: Not with friction and spend alerts. With better defaults, routing, and caching,” Armstrong said.

Coinbase moved engineers to cheaper defaults through an internal LLM gateway rather than lowering caps, reasoning that 91% of employees were never hitting their usage caps to begin with, which meant caps were solving a problem only 9% of the organisation had. Engineers remain free to select any model they judge appropriate.

Coinbase preprocesses prompts in its custom harnesses and routes to the best model for each job, considering cache hits and pricing, on the principle Armstrong described as wanting a frontier model for planning but not for execution, where it is overkill. Cache hit rates rose from 5% to 60%. The price gap the routing exploits is substantial, with reports putting GLM 5.2 at around $1.40 per million input tokens against roughly $5 per million for Anthropic's Opus tier.

Armstrong said the objective was not to suppress usage but to build infrastructure capable of supporting exponential growth in AI workloads while keeping costs under control, and he expects 80% of workloads to run on models 99% cheaper than today's frontier within 12 to 18 months.

The choice of Chinese-origin open-weight models raises governance questions for a federally registered financial firm, and Coinbase's answer to the data-routing concern is self-hosting the open weights on its own servers so that no query data travels to a Chinese API endpoint. That debate remains live and unresolved, though the cost mechanics underneath it do not depend on how it resolves.

The discipline that matters is knowing when not to call the model

The most expensive assumption in enterprise AI is that a frontier model is required at every step. Analyst coverage through 2026 has converged on model routing as the primary optimisation tool, where a routing layer classifies incoming queries by complexity and directs summarisation, classification, extraction and formatting to small cost-optimised models while reserving frontier models for genuine reasoning and generation.

Nigam said the teams doing this well have done the node-by-node work rather than applying a blanket policy across the pipeline. “The teams building agentic pipelines efficiently right now are the ones that have mapped exactly which nodes in their pipeline genuinely require a frontier model and which do not,” he said.

He added that much of what presents as an AI decision is not one at all, since a lot of it is a classification or a lookup that a much cheaper model, or no model at all, can handle. “The discipline of knowing when not to call the model is becoming as important as knowing how to prompt it,” Nigam said.

Ramamoorthy said the same principle governs the infrastructure layer, where the assumption of GPU dependency usually goes unexamined. “Cost efficiency has to be a design decision made early, in the choice of model size and the infrastructure it runs on,” he said.

He explained that Zoho right-sizes the model to the task and runs smaller models efficiently on standard CPUs wherever it can, so the economics still hold when the pilot becomes production. That headroom is measurable, since industry analyses place GPU utilisation during inference operations at 15% to 30%, meaning a substantial share of the GPU budget pays for hardware doing nothing productive.

One team that audited token usage and routed simpler subtasks to cheaper models cut monthly API costs from $40,000 to $24,000 without making any product changes.

The industry is building institutions around that discipline. The Linux Foundation announced the Tokenomics Foundation in June 2026, modelled on how the FinOps Foundation standardised cloud cost discipline a decade earlier.

The FinOps Foundation's 2026 data shows 78% of FinOps practices now report into the CTO or CIO organisation, up 18 points from 2023, with only 8% reporting to the CFO, which classifies AI cost governance as an architecture capability rather than a finance one and reflects where the decisions that determine the bill are actually taken. Survey data cited in coverage of the Uber overrun found that only 43% of organisations have formal AI governance policies and only 21% have mature agentic governance, and that gap is where the next round of invoice surprises will be written.

Sommer's warning to CIOs concerned the balance rather than the direction of travel. “You can't just coast the wave of low-value generative AI, nor can you coast the wave of everything at the frontier,” he said. “If you're constantly moving toward the frontier, your token costs are going to explode to such an extent that you won't be able to recognize a profit at any time.”

What to take away

The token cost crunch is an architecture problem wearing a finance problem's clothing, which is why it will not be solved by the people currently being asked to solve it. Prices have already fallen roughly 280-fold for a fixed capability tier since late 2022, according to the Stanford HAI AI Index, and the bills went up regardless, so waiting for the curve to arrive is not a plan.

Uber and Coinbase ran the same experiment and got opposite results, and the variable separating them was neither discipline nor budget size nor engineering quality, but the point in the sequence at which governance arrived. Uber reached for caps once the money was already gone, and a cap applied at that stage can only describe a problem that has finished happening. Coinbase built defaults, routing and caching into the gateway before the spend compounded, and halved the bill without asking a single engineer to use less.

The practical version is one question, asked before the first production request rather than after the first invoice: which specific step in this workflow genuinely requires the most expensive model, and what is the evidence for it? Most organisations that pull thirty days of calls and sort them by real complexity find that the majority ran on the priciest available model for work that a cheaper one handles identically.

That audit takes about a week, the savings are permanent, and the alternative is learning the same information from an invoice, by which point the architecture is already built, and the choice is no longer cheap to make.