Updates to the vLLM component

rh-aiservices-bu / llm-on-openshift

Resources, demos, recipes,... to work with LLMs on OpenShift with OpenShift AI or Open Data Hub.

Apache License 2.0

90 stars 86 forks source link

Closed carlmes closed 5 months ago

carlmes commented 5 months ago

Updated the following:

Fixed the REST (curl) examples for calling chat and completions endpoints in the README
Added kustomize as an option for manually installing
Added instructions for implementing the Hugging Face hub token requirements that are new as of today

carlmes commented 5 months ago

I redid this merge request, something was wrong with the folder structure in the previous one: https://github.com/rh-aiservices-bu/llm-on-openshift/pull/53

The previous pull request contains curl REST examples showing manual testing on a Red Hat demo cluster.