Add libnccl package to vllm container

rh-aiservices-bu / llm-on-openshift

Resources, demos, recipes,... to work with LLMs on OpenShift with OpenShift AI or Open Data Hub.

Apache License 2.0

90 stars 86 forks source link

Add libnccl package to vllm container #58

Closed bbrowning closed 5 months ago

bbrowning commented 5 months ago

I'm unable to serve multi-gpu models with this container today because it's missing libnccl.so.2. This change installs the missing libnccl, with a version that matches our libnccl-devel package.

I built and pushed an image containing this change to quay.io/bbrowning/vllm-openai-ubi9:0.4.0.post1-1 to verify this fixes the missing libnccl.so.2 issue I encountered.

Fixes #57