text-generation-webui as a platform for serving LLMs

redhat-et / foundation-models-for-documentation

Improve ROSA customer experience (and customer retention) by leveraging foundation models to do “gpt-chat” style search of Red Hat customer documentation assets.

Other

24 stars 11 forks source link

text-generation-webui as a platform for serving LLMs #47

Open codificat opened 1 year ago

codificat commented 1 year ago

Explore text-generation-webui to serve language models on ODH.

How flexible is it for different models?

Would we need a custom server.py for some models?

Shreyanand commented 3 months ago

@codificat can we close this? AFAIK, we had done this?