Build the text generation app from a CUDA base image

redhat-et / foundation-models-for-documentation

Improve ROSA customer experience (and customer retention) by leveraging foundation models to do “gpt-chat” style search of Red Hat customer documentation assets.

Other

24 stars 11 forks source link

Build the text generation app from a CUDA base image #43

Closed codificat closed 1 year ago

codificat commented 1 year ago

The text generation build config was using the standard Python s2i image. This was enough to get things up and running, but not enough for components that expect the CUDA runtime (like bitsandbytes).

This PR updates the BuildConfig to build on top of a CUDA-enabled base image, https://github.com/opendatahub-io-contrib/workbench-images#cuda-base-images

codificat commented 1 year ago

For record, did you select the CUDA image based on any specific hardware version or just the latest image worked?

The latest image JustWorks™