search

ray-project / ray-llm

RayLLM - LLMs on Ray
https://aviary.anyscale.com
Apache License 2.0
1.22k stars 87 forks source link

Request for Comment: RayLLM <-> FastChat Integration #91

Open Extremys opened 10 months ago

Extremys commented 10 months ago

Hello team,

I would like to suggest the idea of the integration with the FastChat project (29k stars). The idea would be to being able to instantiate FastChat compatible workers with RayLLM and take advantage of your both great projects with great features like infra auto-scaling, what do you guys think? It would be amazing to have a such integration!

richardliaw commented 9 months ago

This makes a lot of sense! would you be interested in creating a PR? otherwise this might take some time for us to get to, but we'd be happy to prioritize it.

Extremys commented 9 months ago

I have knowledge of FastChat but not on RayLLM maybe if anybody of the team has a technical presentation/documentation or advises to extend RayLLM :) it would be great! A such integration could also highly contribute to Ray tech adoptions for sure :)