How to quantize models (without killing quality)

Four-bit quantization has a bad reputation for destroying model quality. While it’s true that post-training quantization in 4-bit integer formats makes models noticeably worse, new microscaling data formats like MXFP4 and NVFP4 deliver on the promise of fast low-precision inference without meaningful quality loss. This talk introduces these data formats along with a shift from quantization as a binary decision to quantization as a granular process with model-level considerations (quantization across weights, activations, KV cache, attention) and layer-level considerations (quantization of input, output, and hidden layers) to help you preserve quality while accessing improved performance and cost characteristics from low-precision inference.