This PR adds optional build-arg's for this Dockerfile: comps/llms/text-generation/vllm/docker/Dockerfile.microservice
Issues
Current dockerfile always defaults to gpu Torch installation while CPU users might not necessarily need extra packages.
This leads to smaller containers for CPU users and faster build.
For users interested to do gpu builds all they need to do is build this image this way:
Description
This PR adds optional
build-arg's for this Dockerfile: comps/llms/text-generation/vllm/docker/Dockerfile.microserviceIssues
Current dockerfile always defaults to
gpuTorch installation whileCPUusers might not necessarily need extra packages. This leads to smaller containers for CPU users and faster build.For users interested to do
gpubuilds all they need to do is build this image this way:Type of change
List the type of change like below. Please delete options that are not relevant.
build-argoptionARCHto buildvllm microservicecontainer for either CPU or GPU.Dependencies
None