Speculative Decoding

Overview June 17, 2026

speculative-decoding inference systems

Fast Inference from Transformers via Speculative Decoding — Leviathan et al.
Accelerating Large Language Model Decoding with Speculative Sampling — Chen et al.
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads — Cai et al.
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty — Li et al.
EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees — Li et al.
EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test — Li et al.
Cascade Speculative Drafting for Even Faster LLM Inference — Chen et al.
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding — Fu et al.
Speculative Speculative Decoding — Kumar et al.
DFlash: Block Diffusion for Flash Speculative Decoding — Chen et al.

Overview of speculative decoding methods from foundational draft–verify schemes through EAGLE, Medusa, and diffusion-based drafters, from the NAR LMs Reading Group collection.

Discussion points:

What are the core trade-offs between separate draft models, multi-head drafting, and feature-level speculation?
How do dynamic draft trees and cascaded drafting improve acceptance rates?
Where do diffusion models fit as drafters (e.g., DFlash)?
Which methods are most practical for serving at scale?