Writing
Field notes
- What fits on a 24 GB GPU: measured on an A10G and an L4
Qwen3-4B and Qwen3-8B served with vLLM on two 24 GB GPUs from different generations. Why the 8B model won't start at 32K, why FP8 weights help more than a smaller context, and why both GPUs held the same amount of context.
- A 32B model on a 48 GB GPU: what actually fits
Qwen3-32B on one NVIDIA L40S: why 16-bit weights don’t fit, why 8-bit weights still serve less than one user at 32K, and the two settings that change everything.