Liquid

Multidimensional resource allocation for LLM inference through dynamic parallelism and autoscaling.

August 2023 – August 2024 · Beijing, China

I proposed and implemented Liquid, an autoscaling mechanism for LLM inference built on top of vLLM.

Liquid dynamically adjusts parallelism to allocate resources across multiple dimensions, improving overall resource utilization.