inference-software
v0.32.0
vLLM's new release adds a structured-decisions endpoint, generic parsers, and broader low-precision KV cache support across multiple GPU architectures. It also adds Model Runner features and faster startup and serving improvements.
Sources
- Why it matters
- These changes improve serving performance and reduce the need for model-specific parsers.
- Who is affected
- Developers and operators deploying vLLM, particularly those using structured decisions, low-precision KV caches, or ROCm, are affected.