DeepSeek Open-Sourced Its Inference Engine. Here's What Running It Would Cost You

On October 20 the top of Hacker News read: DeepSeek has open-sourced DeepInfer, the inference engine that serves its own API. Apache 2.0, a tech report, and 200 comments below chewing on two numbers from DeepSeek's Zhihu post. 155 trillion tokens a day from a single cluster. And $38 of revenue per card per hour, at off-peak prices.

The question the thread asks and never answers is the obvious one: if the engine and the model are open, can I just run this myself? And what would it cost me?

We did the math. The short answer: yes, you can. It costs more than the API in every variant. And the interesting part of the release is not what is in it but what is missing.

DeepInfer is the engine behind DeepSeek's production. It grew out of vLLM, the README says so itself, with inspiration from SGLang and TensorRT-LLM. Inside are the kernels DeepSeek released one by one over the last eighteen months (FlashMLA, DeepGEMM, DeepEP), plus speculative decoding (DSpark), asynchronous scheduling, deterministic sampling and structured outputs via JSON schema or grammar.

It supports exactly one model, DeepSeek V4.1 Flash, and exactly two kinds of hardware: NVIDIA SM100, that is Blackwell (B200, B300, GB200, GB300), and Huawei's Ascend 950. No H100, no H200. If you bought Hopper last year you stay on vLLM or SGLang; more on that in a moment.

And then there is the paragraph in the README that should have carried the thread:

V4.1 Flash is a mixture-of-experts model with 552 billion parameters, of which only 8 billion compute per token while reading (prefill) and 16 billion while writing (decode). That makes it fast, not small. The weights ship pre-quantized in FP8 and FP4 and take 510 GB on disk; with headroom the vLLM recipe wants about 614 GB of GPU memory. The license is MIT, so you may do anything with it.

The KV cache, by contrast, is surprisingly small: 890 bytes per token, under a gigabyte at a million tokens of context. The memory problem is the model, not the context.

NVIDIA maintains a deployment recipe for the model. It knows three ways in, and none of them runs over the H100s standing around everywhere.

A Mac Studio with 512 GB stays a hobby. There are reports of 2-bit builds at just under ten tokens per second, and when we wrote this not every mainstream runtime knew the architecture yet.

In October 2026 a B200 rents for between $6.40 and $8.40 an hour depending on the index; we use seven. Eight of them, the whole month, and as throughput the 991 output tokens per second and GPU from the recipe:

Read the last four lines again. Even if your rented node runs flat out around the clock, you pay more per token than DeepSeek charges at daytime prices, and at realistic utilization five times as much. The input side does not help: the API charges a quarter of the output price for input and $0.006 per million for cache hits.

On price alone, renting never wins. For a test, a benchmark or a month of evaluation it is still the right move. For steady operation it is not.

An HGX B200 system with eight GPUs costs between $400,000 and $500,000 in 2026; we take $450,000. A single B200 currently trades at $45,000 to $50,000, trending up. Add power, because the system draws about 14.3 kW.

Against the API at peak prices, buying pays off from about 58 percent utilization, for three years, every day. Against off-peak prices it never does. And all of that before somebody racks the node, maintains the CUDA 13 drivers and picks up the phone at night when a worker hangs.

The numbers are a snapshot, so replace them with your own here:

None of this is a surprise if you know the back story. In March 2025 DeepSeek already showed how its V3/R1 inference ran: H800s at two dollars an hour, $87,000 of cost a day, $562,000 of theoretical revenue. The new figure, $38 per card-hour, is the same idea on better hardware. A card that costs you seven dollars to rent earns DeepSeek five times that. How?

It comes down to three things, and all three are on the list of parts that were not released.

The first is the cache. DeepSeek reports a cache hit rate of 96 percent on input. Agentic workloads look like the NVIDIA benchmark: 64,000 tokens in, 400 out, and of the 64,000 the system has seen most before. A cache hit costs the customer $0.006 per million tokens and costs DeepSeek almost no compute, because the KV cache is loaded from SSD instead of rebuilt. Reports on the release mention 100 PiB of SSD for this cache alone. That is the KV cache system missing from the release.

The second is the split of prefill and decode. While reading, the card computes; while writing, it waits for memory. On separate machines each side can be saturated on its own. In the NVIDIA recipe this buys ten percent more throughput per GPU on B200 and 50 percent more tokens per second per user. For one node in your own rack that is nice. For a cluster where four cards read for 64 cards that write (that is what the DeepInfer README describes), it is the business model.

The third is utilization. 155 trillion tokens a day from one cluster means the cards never idle. Your node, on the other hand, sits empty at night and empty on weekends, and in the sprint review everyone arrives at once. The 58 percent break-even from above is hard to reach for a single company.

The release gives you the engine. It does not give you the economics.

If price is not the argument, the arguments that no price captures remain.

The first is data protection. DeepSeek's own privacy policy says data is stored in the People's Republic of China. For personal data the hosted API is therefore no default. The same weights on a server in Frankfurt are a different matter: model weights are not personal data, and what you send the model never leaves your data center. That is what the comments in the HN thread are getting at when they say European companies are moving to self-hosted or EU-hosted open-weight models right now.

The second is determinism. DeepInfer can run batch-invariant and sample deterministically, on NVIDIA at least. For evals, regression tests and anything that must be documented under the EU AI Act, "same input, same output, tomorrow too" is worth more than it sounds.

The third is control over latency and availability: no rate limit, no off-peak window, no model routed to a new version overnight. DeepSeek currently routes V4 Pro requests to Flash until V4.1 Pro arrives.

What self-hosting does not solve is trust in the model itself. Open weights do not mean you know what is inside. The commenters are right that the output still has to be checked. That applies to every model, though, wherever it runs.

Now for the question that interests us most. If the engine is open and the problem is utilization, the natural solution is not the node in the basement but somebody who pools nodes. A host that puts ten companies on eight cards reaches the utilization that none of the ten reaches alone. And that host sits in Nuremberg, Roubaix or Paris, not in Hangzhou.

What such a host needs is manageable, but not trivial:

• It needs Blackwell. DeepInfer runs only on SM100, and the vLLM and SGLang recipes want at least H200. The H100s standing around everywhere are not enough for this model.
• It needs one node per replica, so eight B200 or four GB200, with 512 GiB of host RAM per group of four and InfiniBand or NVLink between them once prefill and decode run apart.
• It needs a cache layer, and that is the part DeepSeek keeps. vLLM and SGLang bring building blocks (NIXL and Mooncake hand the KV cache from prefill to decode), but the cache that persists for hours and makes the 96 percent possible is its own work.
• It needs metering, a legal entity in the EU (the CLOUD Act follows the company's seat, not the data center's coordinates) and a data processing agreement.

And this is where DeepInfer actually helps, just not the way the thread thinks. Not as a drop-in a host starts tomorrow; the three parts are missing for that. But as a reference: this is what the kernels look like, this is how you decode speculatively, this many cards read for this many cards that write. One commenter on HN reports that inference providers on OpenRouter already serve DeepSeek models faster than DeepSeek's own API. That is the market a host nearby enters. The weights are the same, the engine is now the same, and the difference is utilization, cache and location.

So our answer is this: for most companies the release is no reason to buy hardware. It is a reason to write an email to the host they trust.