ray-project / ray-llm

RayLLM - LLMs on Ray
https://aviary.anyscale.com
Apache License 2.0
1.22k stars 87 forks source link

[docs] Improve docs around configuration #74

Open richardliaw opened 10 months ago

richardliaw commented 10 months ago

We need more polish for the config api, especially the scaling config: • I don’t understand what it is about by looking at this name • it still talks about Ray AIR. • The config itself is not that intuitive. I need to think about it fro a while and then realize that each model replica actor will spawn multiple workers each of which requests those ray resources

Screenshot 2023-10-16 at 3 53 47 PM

akshay-anyscale commented 10 months ago

Some docs improvements here - https://github.com/ray-project/ray-llm/pull/85

YixuanCao commented 8 months ago

+1. I cannot figure out the meaning of these configs. They seem to be inherited from Ray Serve. But I think the ray-llm document should be self-contained.