Closed sangstar closed 7 months ago
Adds support for housing vllm-tensorizer container image on ml-containers. vllm-tensorizer is a container image that runs an OpenAI-compatible API server using vllm along with support for tensorizing models as a load_format.
vllm-tensorizer
ml-containers
vllm
load_format
Adds support for housing
vllm-tensorizer
container image onml-containers
.vllm-tensorizer
is a container image that runs an OpenAI-compatible API server usingvllm
along with support for tensorizing models as aload_format
.