kubeagi / arcadia

A diverse, simple, and secure one-stop LLMOps platform
http://www.kubeagi.com/
Apache License 2.0
63 stars 20 forks source link

Add ray-serve-vllm deployment sample #1020

Closed nkwangleiGIT closed 2 months ago

nkwangleiGIT commented 2 months ago
  1. Add script to run LLM serving with Ray Serve
  2. Add Dockerfile and RayServie yaml to prepare and deploy inference service using CR