Serve a new model without restarting RayLLM

ray-project / ray-llm

RayLLM - LLMs on Ray

https://aviary.anyscale.com

Apache License 2.0

1.2k stars 87 forks source link

Open k6l3 opened 5 months ago

k6l3 commented 5 months ago

Is it possible to start serving a new model when an event happens?

samarth-contextual commented 5 months ago

Bumping this.