search

rh-aiservices-bu / llm-on-openshift

Resources, demos, recipes,... to work with LLMs on OpenShift with OpenShift AI or Open Data Hub.
Apache License 2.0
90 stars 86 forks source link

Enable vLLM for FIPS enabled enviroments #61

Closed maxisses closed 4 months ago

maxisses commented 5 months ago

vLLM Deployment fails on FIPS enabled clusters. Adding the following to the Dockerfile of the vLLM build fixes that issue:

USER 0 RUN sed -i s/md5/sha1/g /opt/app-root/lib64/python3.11/site-packages/triton/runtime/jit.py USER 1001