Open richardliaw opened 10 months ago
Some docs improvements here - https://github.com/ray-project/ray-llm/pull/85
+1. I cannot figure out the meaning of these configs. They seem to be inherited from Ray Serve. But I think the ray-llm document should be self-contained.
We need more polish for the config api, especially the scaling config: • I don’t understand what it is about by looking at this name • it still talks about Ray AIR. • The config itself is not that intuitive. I need to think about it fro a while and then realize that each model replica actor will spawn multiple workers each of which requests those ray resources