Skip to content

INFERENCE LAB / INFERENCE

Inference

Deploy, serve and optimize. Practical notes on the engines behind LLM inference.

All topics vLLM SGLang llama.cpp TensorRT-LLM

Inference

Understanding FP8 KV Cache in vLLM

Understand what KV cache quantization changes and build a compatibility and quality check before enabling it.

vLLM KV Cache FP8

Sep 7, 2026 · 2 min read

Reference guide · awaiting lab validation

Inference

Prefix Cache vs KV Cache in LLM Inference

Distinguish the attention state stored during generation from reuse of that state across requests.

vLLM SGLang KV Cache

Sep 7, 2026 · 2 min read

Reference guide · awaiting lab validation

Inference

Qwen3.8-27B on 4×RTX 4090: Deployment Guide

A deployment validation plan for Qwen3.8-27B, covering hardware inventory, version pinning and a reproducible acceptance test.

Qwen RTX 4090 SGLang

Sep 7, 2026 · 2 min read

Reference guide · awaiting lab validation

Inference

SGLang MTP / NEXTN Explained

Separate speculative decoding concepts from model- and release-specific launch options.

SGLang MTP Speculative Decoding

Sep 7, 2026 · 2 min read

Reference guide · awaiting lab validation

Inference

vLLM vs SGLang: Choosing an Inference Engine

Choose an inference engine using workload compatibility, reproducibility and operational requirements—not an unqualified leaderboard.

vLLM SGLang Inference

Sep 7, 2026 · 2 min read

Reference guide · awaiting lab validation