support vLLM for gaudi by k8s

dell / omnia

An open-source toolkit for deploying and managing high performance clusters for HPC, AI, and data analytics workloads.

https://omnia-doc.readthedocs.io/en/latest/index.html

Apache License 2.0

224 stars 118 forks source link

support vLLM for gaudi by k8s #2308

Closed dweineha closed 1 month ago

dweineha commented 2 months ago

Add support for vLLM

Signed-off-by: Zheng Wang zheng.w.wang@intel.com

priti-parate commented 1 month ago

Closing this PR because approach is different for vLLM deployment. gaudi-llama3.yml file will be delivered as part of examples/ai_examples folder and deployment steps will be documented in omnia docs