Closed sangstar closed 7 months ago
Adds support for housing vllm-tensorizer container image on ml-containers. vllm-tensorizer is a container image that runs an OpenAI-compatible API server using vllm along with support for tensorizing models as a load_format.
vllm-tensorizer
ml-containers
vllm
load_format
Adds support for housing
vllm-tensorizercontainer image onml-containers.vllm-tensorizeris a container image that runs an OpenAI-compatible API server usingvllmalong with support for tensorizing models as aload_format.