← All news

· Antti Pasila · 7 min read

The cookbook is out: what DeepInfer means for the trillion-dollar data center bet

DeepSeek open-sourced how it serves 155 trillion tokens a day. If inference becomes a commodity, the data center build-out has a pricing problem.

Last week DeepSeek open-sourced DeepInfer, the inference framework behind its own production service. I wrote a short take on LinkedIn calling it the first domino in the AI data center story. A few people asked me to show the math. This is the long version, with the numbers, the strongest counterargument, and what I think it means if you are building with AI or paying for it.

My claim, in one line: when the cost of serving a good-enough model drops this far, and buyers stop caring which brand answers their prompt, the price of a token falls faster than demand grows. That is a problem for anyone who borrowed money to build general-purpose capacity at today's prices.

What DeepSeek actually released

DeepInfer is the engine DeepSeek uses to serve DeepSeek-V4.1-Flash. It runs on NVIDIA's current Blackwell GPUs and on Huawei's Ascend 950 chips, and it is released under the Apache 2.0 license, so anyone can use it commercially. Alongside it, DeepSeek published a detailed write-up of how its production system works, with numbers most labs treat as trade secrets.

The headline figures: a single DeepSeek cluster serves 155 trillion tokens a day, and it earns about $38 per GPU per hour at off-peak prices. The average request carries about 77,000 input tokens, and 96% of those are served from cache rather than computed again.

To be precise about what is in the box: the open release is the engine, the optimised kernels and the parallelism examples. The pieces that turn it into a full service, such as the shared cache, the split between the stages that read prompts and write answers, and the load balancers, are not included yet. So this is the cookbook, not the restaurant. But it is the cookbook of the most efficient kitchen in the business, and the missing pieces are well understood engineering, not secret science.

One more detail deserves attention. Support for Huawei's Ascend 950 means a frontier-class serving stack no longer depends on NVIDIA. That matters for China, and for any country that wants its own AI capacity without waiting in NVIDIA's queue.

What a token really costs

DeepSeek describes its efficient serving unit as 64 GPUs reading prompts and 64 GPUs writing answers. Using the per-GPU throughput it reports, here is what that unit handles in a day.

One serving unit, one day

Input read, including cached context8.3T tokens
New input actually computed332B tokens
Output generated77B tokens

One 128-GPU serving unit (64 prefill + 64 decode) at DeepSeek's reported per-GPU throughput, running 24 hours.

Most of the volume is the model re-reading context it has already seen. Agents send their whole history with every step, and caching turns that into a cheap memory read.

Source: DeepSeek DeepInfer production figures; Cyans calculation.

This is why both of the numbers I have used are right, depending on what you count. Counting everything the unit reads, it handles over 8 trillion tokens a day. Counting only the work it actually computes, it is about 400 billion. Either way, the scale per machine is remarkable.

What does the unit cost? Reading prompts happens on standard 8-GPU servers and writing answers on a 72-GPU rack. Industry quotes put that rack at about $6 million, so with eight servers, networking and storage you land around $10 million. That is our estimate, since NVIDIA does not publish prices.

At DeepSeek's reported $38 per GPU-hour, that $10 million unit brings in about $42 million a year, at off-peak prices. Power for a unit like this costs a few hundred thousand dollars a year. Even after networking, staff and a generous margin for error, the hardware pays for itself in months, not years.

When one operator can cut its price by five times and still make money, that operator sets everyone else's price.

That is the point. These figures don't say inference is a bad business. They say an efficient operator has an enormous amount of room to cut prices, and now everyone has the recipe to become one.

Intelligence is becoming fungible

A commodity is something buyers can swap without caring who made it. For tokens, you can watch that happening in real time on OpenRouter, the marketplace where developers route requests to whichever model they choose.

Tokens a day on OpenRouter

0T5T10T15T20T25TJan 25Jan 26Oct 26

Monthly averages. Traffic is up 36 times on the same week a year ago.

Source: OpenRouter public rankings via the 06.ai tracker, October 2026.

Two things stand out. The first is volume: OpenRouter alone now handles 25 to 30 trillion tokens a day. The second is who serves them.

Who serves the tokens, and who gets paid

Share of tokens served

62%
24%
14%
  • Chinese models 62%
  • US 24%
  • Other 14%

Share of dollars spent

63%
37%
  • US models 63%
  • Everyone else 37%

Over the past four weeks. Chinese models carry most of the work. US models still collect most of the money.

Source: 06.ai OpenRouter tracker, four weeks to October 28, 2026.

DeepSeek V4.1 Flash on its own carries about a quarter of all OpenRouter tokens. Developers are voting with their traffic: for most of the work, a fast, cheap model is good enough. My view, from building agents every day, is that a Flash-class model in a well-built harness, with the right tools, memory and checks around it, handles most office work just fine. The harness matters more than the logo.

The gap between the two bars is the interesting part. It is the premium buyers still pay for brand, habit and the hardest tasks. As more buyers move to routing by price and measured quality, that gap closes.

The strongest counterargument: cheaper means more

The best objection to my argument has a name: the Jevons paradox. When something gets cheaper, people use far more of it, and total spending can go up rather than down. Coal-efficient steam engines did not reduce coal use. They exploded it.

This objection is real, and the OpenRouter chart already shows it happening. Agents consume tokens at a rate no human chatting with a model ever could, and every price cut opens up new uses. I fully expect token demand to keep multiplying.

But investors don't get paid in tokens. They get paid in dollars per GPU-hour. The question is not whether we will use a hundred times more tokens, but whether a hundred times more tokens arrives before the price per token falls a hundred times. The share chart is the early warning: volume is moving to the cheapest capable models much faster than spending is.

Data centers are priced on long contracts and multi-year depreciation. If the revenue per GPU-hour that justified a build in 2025 halves by 2028, demand growth doesn't rescue the loan. It just means the cheap operator next door is busy.

The trillion-dollar math

What is being spent, and what has to come back

Hyperscaler capex, 2026$800B
Hyperscaler capex, 2027 (Goldman Sachs estimate)$1.2T
AI spend needed for solid returns, per year$1T
AI revenue needed just to break even, per year$300B
Everything bought through OpenRouter, per year$4.9B

US hyperscaler figures from Goldman Sachs; OpenRouter spend is annualised from public rankings.

Goldman Sachs estimates hyperscalers need about $300 billion a year in AI revenue to break even, and about $1 trillion a year in end-user AI spending for solid returns.

Source: Goldman Sachs US Equity Views, 2026; 06.ai OpenRouter tracker.

The largest US cloud companies are on track to spend about $800 billion on capital this year, and consensus has them above $1 trillion next year. Add OpenAI's Stargate project and the wider group of global cloud providers and the total is well past a trillion.

Now a thought experiment. DeepSeek's single cluster serves 155 trillion tokens a day, five to six times everything that flows through OpenRouter's public rankings. At its reported per-GPU throughput, running flat out, that takes roughly 2,400 GPUs. Even with three times the headroom for peaks and failures, that is a few hundred million dollars of hardware. Token counting differs between platforms, so treat this as an order of magnitude, not a precise ratio. But the order of magnitude is the story: the general-purpose inference that most people mean when they say "AI" does not need a trillion dollars of new buildings.

Who gets squeezed, and who wins

The squeeze lands on businesses whose only advantage is access to GPUs. Neoclouds that financed fleets with debt and rent them by the hour. Inference resellers with no product beyond an API key. Developers building speculative capacity on the assumption that today's rental rates hold for five years. When the recipe for running efficiently is free, scarcity stops being a moat.

The winners are the buyers, first of all. Enterprises paying per token will see prices fall and choice grow. Next come the builders of harnesses and products, because when the model is a commodity, the value moves to what you do with it. Then efficient operators with cheap power. And finally chip diversity: a serious serving stack on Ascend means NVIDIA's pricing power has a ceiling it did not have last year.

Why the frontier labs still want the clusters

None of this means the giant clusters are pointless. Training the next frontier model still takes enormous concentrated compute. So do scientific workloads such as protein folding, digital biology, materials and climate simulation, along with video and world models. Those are real reasons to build, and some of that work will matter enormously.

There is also a less flattering reason. When training, research and inference all sit in one giant budget, the unit economics get murky. Nobody has to show what a token costs to serve, or what each new data center earns. That suits the labs. It suits the public much less, because pension funds, bond buyers and electricity ratepayers are financing much of this build-out, and they are underwriting an inference business that may be priced like a commodity within a few years.

What I would do as a buyer today

Don't lock in long commitments at today's per-token prices. The curve is moving in your favour.

Build a model-agnostic harness. If switching models is a config change rather than a rewrite, you can take every price cut as it comes. This is exactly why we built Autark: one OpenAI-compatible API that routes every request to the best open model, chosen by continuous blind evaluations.

Test Flash-class models on your own work. Blind-judge them against the premium models on real tasks from your backlog. Most teams find the gap is smaller than the price difference.

Design for caching. Keep the stable parts of your prompts at the start, such as system instructions, tool definitions and reference documents. Cached input is the cheapest token there is, and agents that reuse context well cost a fraction of the ones that don't.

DeepSeek didn't release a product last week. It released a price signal. The companies that read it early will buy intelligence cheaply for years. The ones that built on the assumption it would stay expensive are about to find out what a commodity cycle feels like.

Sources

  1. DeepInfer on GitHub (DeepSeek-AI)
  2. SiliconANGLE: DeepSeek open-sources DeepInfer (October 20, 2026)
  3. DeepInfer, explained: production design and figures
  4. How a request flows through DeepInfer, production vs open source
  5. 06.ai OpenRouter tokens and spend tracker
  6. Tidelines OpenRouter rankings
  7. Tom's Hardware: NVL72 and Vera Rubin rack prices
  8. Goldman Sachs: AI capex, required revenues and what's priced in
  9. Yield Theory: hyperscaler capex 2027 forecast
  10. TrendForce: CSP capex projected to exceed $1.5 trillion in 2027