search

lm-sys / RouteLLM

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality!
Apache License 2.0
3.3k stars 250 forks source link

Dockerizing Route-LLM 🚀 #33

Open szymonrucinski opened 4 months ago

szymonrucinski commented 4 months ago

Hi there! 👋

I've been following the Route-LLM project and I must say, it's really cool! The idea of using language models to generate routes and directions is brilliant. 🌟

However, I noticed that getting started with the project might be a bit confusing for some users, especially those who are new to the tech stack used. 🤔

I believe that deploying the project using Docker could make it even more accessible and user-friendly. By providing a Dockerfile and docker-compose.yml file, anyone interested in using Route-LLM could quickly build and run the server within seconds. This would greatly simplify the setup process and reduce the chances of running into environment-related issues. 🐳

Dockerizing the project could also help increase its popularity and adoption. Many developers and users prefer projects that are easy to deploy and run, and Docker has become a standard for containerization. 📈

To contribute to this effort, I have created a Dockerfile and docker-compose.yml file that document the workflow and make it straightforward to deploy Route-LLM on the cloud or any Docker-compatible environment. 🚀

I hope this contribution helps make Route-LLM even more accessible and attractive to a wider audience. Let me know if you have any feedback or suggestions!

Keep up the great work! 😊

bitnom commented 3 months ago

I have improved it and included cuda. Just got it working so still testing and need to make a docker-compose version