search

redhat-et / foundation-models-for-documentation

Improve ROSA customer experience (and customer retention) by leveraging foundation models to do “gpt-chat” style search of Red Hat customer documentation assets.
Other
24 stars 11 forks source link

Serving LLMs on Open Data Hub / RHODS with ModelMesh #46

Open codificat opened 1 year ago

codificat commented 1 year ago

How can we create model endpoints using Modelmesh on the cluster?

This should involve instructions on deploying modelmesh using Operator Hub and adding the model to a s3 bucket.

Create the predict function and explain how it is being called and demo a notebook that sends a payload to the model endpoint.

Shreyanand commented 3 months ago

@codificat can we add the document that you created to the repo for the record?