From DSpark to Disciplined Agentic AI
What a database history lesson says about how much engineering is still left in AI infrastructure
June 2026 — Chandra Pendyala
| ABSTRACT: DeepSeek’s DSpark is useful as more than a speedup number. Its gain comes from a small structural addition outside a frozen, pretrained model: draft cheaply, estimate survival, and verify only what has earned expensive target-model compute.That is the pattern AI infrastructure now needs above the token layer. New empirical work on agentic systems shows the missing optimizer: token consumption three orders of magnitude above ordinary use, run-to-run variance up to 30x, and models unable to predict their own execution cost.The economic question is not cost per token. It is how much work crosses from theoretically capable to reliably, cheaply automatable. That frontier is still constrained less by intelligence than by execution discipline. |
A Borrowed Calibration
SQL gave databases a powerful interface in the early 1980s — declare what you want, not how to get it. It took another decade and a half before query optimizers, indexes, and cost-based execution made that interface cheap at scale. The gap wasn’t a single breakthrough; it was years of unglamorous engineering — estimating selectivity, pruning unnecessary scans, learning to skip work rather than just do it faster. By 2025, Alibaba’s PolarDB was reporting 2.055 billion tpmC at 0.80 CNY/tpmC — roughly $0.11 per unit of TPC-C price/performance[1] — a result made possible not only by faster hardware, but by decades of systems engineering that taught databases to avoid, parallelize, and amortize unnecessary work.
That gap — between a powerful interface and a disciplined execution layer underneath it — is where AI infrastructure sits today. Chat and agentic tool use play a role analogous to SQL in this era: a genuinely powerful way to ask a system to do something, sitting on a substrate that is still mostly brute force. Call the distance between the two the optimizer gap: the space between a system that can express or attempt useful work and one that can execute that work with disciplined allocation of compute, context, tools, verification, and retries. This paper looks at one place where that gap has started to close, what its closing actually teaches, and how wide it remains everywhere else.
Section 1 — DSpark: Proof the Discipline Is Achievable
DeepSeek’s DSpark, published with DeepSeek-AI and Peking University researchers in June 2026, rewards close reading not for its headline speedup but for the engineering underneath it — a clean instance of cost-disciplined execution done correctly.[2]
The problem it solves
Autoregressive generation produces one token per forward pass, so latency scales with output length. Speculative decoding breaks this by having a cheap draft model propose several tokens at once, which the expensive target model then verifies in a single pass — accepting the longest prefix consistent with its own distribution.[2] The economics reduce to one equation: latency per token equals (draft time + verify time) divided by tokens accepted.[2] Three levers exist: draft faster, draft better, or verify smarter.
Prior approaches forced a tradeoff between the first two. Autoregressive drafters condition each token on the ones before it — strong quality, but drafting cost scales linearly with block size, so they’re stuck with short, shallow drafts. Parallel drafters generate an entire block in one pass, decoupling speed from block size, but because each position is predicted independently, they can’t condition on what was actually sampled elsewhere in the block. The result is multi-modal collision: a model that might plausibly say “of course” or “no problem” independently proposes “of problem.”[2] DSpark’s own measurements show this concretely — on open-ended chat, a parallel drafter’s position-level acceptance rate falls from roughly 0.72 at the first token to 0.63 by the seventh, while an autoregressive drafter trends upward over the same span, because each token can condition on the one before it.[2]
The fix: a little autoregression, deliberately placed
DSpark’s answer is semi-autoregressive generation: keep the expensive backbone fully parallel, then append a deliberately tiny sequential module that injects just enough dependency to stop the suffix decay. The default is a low-rank Markov head: given the previous draft token, it boosts and suppresses logits at the next position via a rank-256 factorization, cheap enough that the sequential loop stays negligible relative to the parallel pass.[2] The empirical result is the most instructive number in the paper: a 2-layer DSpark drafter outperforms a 5-layer pure-parallel baseline across every domain tested.[2] Not a speed claim — a parameter-efficiency claim. A small, well-placed piece of sequential structure beats brute-force depth.
Verifying smarter, not just drafting better
A longer, higher-quality draft block doesn’t automatically translate into a faster system, because verifying it still costs scarce target-model compute, and the value of verification varies enormously — by domain and by system load. In DSpark’s threshold sweep, pruning low-confidence suffixes raises Code acceptance from 67.6% to 92.0%, Math from 76.9% to 92.5%, and Chat from 45.7% to 95.7%, while an extra verified token is nearly free when idle but expensive when the system is saturated.[2] DSpark’s answer is a confidence head that estimates, per draft position, the probability the token survives verification given everything before it survived — and a scheduler that uses those estimates to decide, per request and per moment, how much of the draft block is worth sending to the target model at all.[2]
Correctness as the actual constraint
A naive version of this scheduler would rank every candidate token across every active request by survival probability and admit the top ones — a clean combinatorial optimization. DSpark’s authors explicitly reject a retrospective global search because future sampled-token information can influence an earlier admission decision, violating the non-anticipating property speculative decoding depends on for exact target-distribution recovery.[2] In production, the scheduler approximates upcoming verification capacity using confidence estimates from two steps prior rather than the current step, trading a small amount of staleness for pipeline compatibility and for never letting an admission decision peek at information it should not have.[2] This is the detail worth dwelling on, because it generalizes better than the throughput number does: a cost-saving mechanism is only sound if its savings come from genuinely not needing certain work, not from quietly cutting a correctness corner. That bar — selective without being unsound — is the actual definition of cost-based intelligence, and it’s higher than most agentic tooling today even attempts to clear.
Deployed against DeepSeek-V4’s production traffic, replacing the prior single-token MTP baseline, DSpark accelerated per-user generation by 60–85% on V4-Flash and 57–78% on V4-Pro at matched throughput levels, with the largest reported multiples — near each model’s interactivity ceiling — better read as evidence the baseline was collapsing there than as a representative speedup.[2] The actual contribution is a serving system that decided, continuously and per request, how much expensive verification a given draft block had earned.
The Win Wasn’t Pretraining
The target model in DSpark is frozen throughout. Nothing about pretraining changed. The entire gain comes from a small structural addition outside the model: a low-rank Markov head, a confidence estimator supervised against an analytically exact target rather than a heuristic, and a scheduler that earns the right to prune work only by proving it isn’t leaking future information. The cleanest evidence this is a structural win, not a scale win, is the depth result: a 2-layer DSpark drafter beats a 5-layer parallel-only baseline. More parameters lost to a smaller, better idea about the problem’s actual shape — not a repudiation of scale, but a reminder that once scale exposes a useful substrate, structural knowledge about the task can beat indiscriminate depth.
That reflex was earned, but it answered a narrower question than the one we’re stuck on now. Scaling laws made next-token prediction economically predictable enough to aim compute at; they said nothing about agentic execution, which is why chat got good while reliability didn’t. Pretraining at this scale wasn’t an accident — the field had a curve to aim compute at years before it had a real theory of why the curve worked, the same gap Codd lived through: a correct declarative abstraction for relational data more than a decade before anyone had a cost-based theory of executing it cheaply. The theory arrived, in both cases, under pressure from people who had to make the thing run. That’s the position infrastructure is in now. Brute force bought a frozen substrate that pattern-matches astonishingly well; it didn’t buy execution discipline, and DSpark shows the two are separable — you don’t retrain the model, you characterize the waste and build something narrow enough to remove it without breaking correctness. The real test isn’t benchmark accuracy. It’s how much work crosses from theoretically capable to reliably, cheaply automatable — and that number is still small relative to the capability headline. Scaling moved the capability headline much faster than it moved the frontier of reliably, cheaply automatable work.
Section 2 — The Rest of the Stack Hasn’t Caught Up
Decoding was the easiest layer to discipline: a single, well-defined per-token decision, a clean mathematical structure, and a tight feedback loop between confidence estimate and ground truth. Agentic execution has none of those properties — the decision to make another tool call, re-read a file, or run another review pass is distributed across an entire trajectory, with no equivalent of a profiled cost table to consult. Three recent empirical studies size what that absence costs in practice.
Bai et al. (2026) ran the first systematic study of where agentic coding systems actually spend tokens, analyzing trajectories from eight frontier models on SWE-bench Verified.[3] Agentic coding tasks averaged 4.17 million tokens and $1.86 per task, against roughly 3,400 tokens for code chat and 1,200 for single-turn code reasoning — a gap on the order of 1,000x to 1,200x, driven overwhelmingly by input tokens rather than output.[3] Token usage on identical tasks varied by up to 30x run to run, and more tokens did not reliably buy more accuracy — success typically peaked at intermediate spend and flattened or degraded beyond it.[3] Token efficiency varied sharply by model even on the same tasks: two frontier models consumed over 1.5 million more tokens on average than the most efficient model tested, for comparable outcomes.[3] Most telling for anyone trying to build cost-aware systems: when the models were asked to predict their own token cost before execution, correlation with actual cost topped out at 0.39, with systematic underestimation.[3] A system that can’t estimate its own cost can’t be made to schedule around it — precisely the capability DSpark’s confidence head provides at the token level and agentic frameworks largely lack at the action level.
AgencyBench moved evaluation toward realistic, long-horizon work: 138 tasks across 32 scenarios drawn from real agentic use cases, each requiring an average of 90 tool calls, roughly one million tokens, and hours of wall-clock execution.[4] It also surfaced a gap that compounds the cost problem rather than offsetting it: closed-source models led open-source models by a wide margin (48.4% versus 32.1% success) — the systems currently best able to complete these trajectories are generally the ones enterprises access through higher-cost frontier APIs.[4]
Salim et al. (2026), studying a ChatDev multi-agent software-engineering pipeline on a GPT-5 reasoning backbone, mapped token consumption onto standard development lifecycle phases.[5] The iterative Code Review stage alone accounted for an average of 59.4% of all tokens consumed across 30 tasks — more than design, coding, completion, and testing combined.[5] Input tokens dominated overall, at 53.9% of consumption on average, with review and documentation phases running as input-heavy as 51–80%, evidence of a “communication tax”: agents repeatedly re-loading large contexts to review work rather than generating new work.[5][6] The paper’s own framing is the cleanest summary available: the primary cost of agentic software engineering is not initial generation. It is refinement and verification, run again and again with no apparent budget.
| Study | What it measured | Headline finding |
|---|---|---|
| Bai et al., 2026 [3] | 8 frontier models, SWE-bench Verified | ~1,000x more tokens than code chat/reasoning; 30x run-to-run variance; self-predicted cost correlates at only 0.39 with actual cost |
| AgencyBench, 2026 [4] | 138 long-horizon tasks, 32 scenarios | ~90 tool calls and ~1M tokens per task on average; closed-source models lead open-source 48.4% to 32.1% |
| Salim et al., 2026 [5] | ChatDev, 30 SE tasks, GPT-5 backbone | Code review alone consumes 59.4% of tokens; input tokens average 53.9% of total consumption |
None of this means agentic systems are incapable — the same body of work documents real task success at meaningful scale. It means the execution layer underneath that capability is still, structurally, doing the equivalent of a full table scan for every query: re-reading full context on every review pass, re-verifying work that hasn’t materially changed, retrying without any cost-aware policy for when a retry is worth its price. DSpark shows this is solvable at the token level, by structural insight rather than more pretraining. Nothing in the current agentic stack shows it solved at the action level yet.
Section 3 — Why the Gap Matters Beyond the Bill
The direct consequence of closing this gap is cost, but not the only one. Every token an agent burns on indiscriminate review, redundant context reloading, or an unbudgeted retry is a token — and a unit of latency, and a unit of risk — that isn’t available for the verification the task actually needs. A system with a real cost-of-action model, the agentic equivalent of DSpark’s confidence head, can afford to spend more on the review pass that’s genuinely uncertain precisely because it isn’t spending uniformly on every pass regardless of need. Efficiency at the model and infrastructure layer is the budget reliability spends from — the less work has to be manually rationed by an application’s harness, the more of that discipline can live where it belongs, in the execution substrate itself rather than bolted on per deployment.
Where This Leaves the Industry
Databases took roughly fifteen years to go from a powerful declarative interface to a cost-based execution engine that mostly stopped doing unnecessary work, and another two decades to compound those gains into systems reporting 2.055 billion tpmC at roughly $0.11 per TPC-C price/performance unit.[1] DSpark is evidence the same transition is underway in AI infrastructure — and that it’s tractable through engineering, not another round of scale. The agentic evidence is the honest accounting of how much of that engineering hasn’t happened yet, and of how much of the economy’s work is still sitting on the wrong side of the line between theoretically capable and reliably, cheaply automatable. The decoding layer got its optimizer first because it was the easiest layer to give one. Planning, tool use, and verification are harder, more distributed problems — exactly why they’re where the next several years of this work need to go.
The work ahead is not merely better models; it is the execution layer that decides when not to use them.
DSpark prunes speculative tokens. The next systems have to prune speculative work.
References
[1] Transaction Processing Performance Council. “TPC-C Benchmark Report: Alibaba Cloud PolarDB Limitless.” TPC Full Disclosure Report, report date January 19, 2025. See also Alibaba Cloud, “Alibaba Cloud’s PolarDB Breaks TPC-C Benchmark World Record with Innovative Three-Layer Decoupling Architecture,” Alibaba Cloud Press Room, February 26, 2025.
[2] Cheng, X., Yu, X., Shao, C., Li, J., Xiong, Y. et al. (DeepSeek-AI, Peking University). DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation. DeepSpec technical report, June 2026. https://github.com/deepseek-ai/DeepSpec/blob/main/DSpark_paper.pdf
[3] Bai, L., Huang, Z., Wang, X., Sun, J., Mihalcea, R., Brynjolfsson, E., Pentland, A., Pei, J. How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks. arXiv:2604.22750, 2026.
[4] Li, K., Shi, J., Xiao, Y., Jiang, M., Sun, J., Wu, Y., Fu, D., Xia, S., Cai, X., Xu, T., Si, W., Li, W., Wang, D., Liu, P. AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts. arXiv:2601.11044, 2026. ACL 2026.
[5] Salim, M., Latendresse, J., Khatoonabadi, S. H., Shihab, E. Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering. arXiv:2601.14470, 2026. MSR ’26.
[6] Wang, Q., Tang, Z., Jiang, Z., Chen, N., Wang, T., He, B. AgentTaxo: Dissecting and Benchmarking Token Distribution of LLM Multi-Agent Systems. ICLR 2025 Workshop on Foundation Models in the Wild, 2025.
Statistics are sourced from primary technical reports, arXiv preprints, benchmark papers, and workshop publications; peer-review status varies by source. Working paper — June 2026.