Add a Job to pre-populate a model into the web UI's storage

redhat-et / foundation-models-for-documentation

Improve ROSA customer experience (and customer retention) by leveraging foundation models to do “gpt-chat” style search of Red Hat customer documentation assets.

Other

26 stars 12 forks source link

Add a Job to pre-populate a model into the web UI's storage #42

Closed codificat closed 1 year ago

codificat commented 1 year ago

The text-generation-webui does not start unless there is at least one model available.

The instructions we have cover how to populate the PVC manually, but this is clunky - at least for the initial model (the pod keeps restarting, you get to fight to grab access to the PVC, etc).

This PR adds a Job definition that downloads the first model. Both the Deployment and the Job have this hardcoded to bloom 1.7B.

codificat commented 1 year ago

Also updating the deployment, service and route(s) to enable different paths for different targets: web UI, REST API, streaming API