SGLang MTP / NEXTN Explained
Separate speculative decoding concepts from model- and release-specific launch options.
Problem
A launch command mentioning MTP or NEXTN is easy to copy but may refer to a different model architecture, backend or release.
Conclusion
Speculative decoding uses candidate generation followed by target-model verification. MTP support and algorithm labels are implementation-specific. Check the exact model and SGLang release documentation; this lab has not validated a universal NEXTN command.
Environment
Record target checkpoint revision, any draft checkpoint, SGLang commit, GPU backend, precision and all speculative settings. Keep a non-speculative baseline with otherwise matching settings.
Symptoms
A configuration may be rejected, use an unsupported path, or add overhead that outweighs accepted draft tokens. No measured acceptance rate or speedup is available here.
Cause
Drafting and verification both cost time. The balance depends on the model, request distribution, concurrency and how many proposed tokens are accepted. The name of an algorithm is not a guarantee of acceleration.
Solution
First confirm ordinary decoding works. Read the speculative-decoding section for the pinned release and the selected model. Enable only the documented compatible path. Change one speculative setting at a time and keep an experiment log.
{
"status": "not_run",
"baseline": { "speculation": false },
"candidate": { "algorithm": null },
"target_revision": null,
"draft_revision": null,
"acceptance_metrics": null
}
Compare end-to-end latency and throughput as well as acceptance behavior. Include drafting overhead and failed requests. Keep correctness and sampling conditions aligned with your application.
Verification
Benchmark data pending real-world testing.
Publish a recommendation only with the complete launch commands, raw measurements and stated compatibility scope.
Caveats
MTP and a separate draft model are not interchangeable configuration choices. NEXTN naming in one backend or release should not be generalized to all SGLang deployments.