LLM Inference in 100 Questions: Serving & Optimization

Every time you send a prompt to a chatbot, a GPU somewhere has to load billions of weights, juggle hundreds of other users and stream your answer back token by token, fast enough to feel instant and cheap enough to make business sense. This course explains how that actually works. LLM Inference in 100 Questions is a complete, visual tour of large language model serving and optimization, organized as 100 short, focused lessons of about 3 minutes each.

Every lesson answers one concrete question, with animated diagrams, worked numbers and a clear takeaway. What you will coverHow a model generates an answer: tokens, attention, prefill vs. decode, the KV cache and streamingHardware and bottlenecks: GPU spec sheets, HBM vs.

SRAM, NVLink, memory sizing, arithmetic intensity and the roofline modelMeasuring and costing: TTFT, ITL/TPOT, throughput, Little's law, benchmarking and the cost of a million tokensBatching: static, dynamic and continuous batching, chunked prefill, schedulers and preemptionAttention and memory: MQA, GQA, MLA, FlashAttention, PagedAttention and KV cache quantizationShrinking the model: FP16, BF16, INT8, FP8, FP4, GPTQ, AWQ, distillation and sparsityNever doing the same work twice: prefix caching, RadixAttention, semantic caching and speculative decoding (Medusa, EAGLE)Serving architecture: load balancing, autoscaling, tensor/pipeline/expert parallelism, disaggregated prefill and decode, cache-aware routingEngines and platforms: vLLM, SGLang, TensorRT-LLM, llama. cpp, managed cloud options, build vs. rentWhat's next: multimodal serving, multi-LoRA, edge inference, semantic routing, RL and new acceleratorsThe course is based on the book "100 Soruda Büyük Dil Modelleri" (Large Language Models in 100 Questions) by Buğra Ayan.

The lessons are narrated by the author. English subtitles are included. No GPU is required to follow along.

By the end, you will be able to reason about latency, throughput and cost like an inference engineer, and choose the right optimizations for your own workload.