lm-sys / RouteLLM

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality!
Apache License 2.0
3.3k stars 250 forks source link

Ability to Incorporate Open Source Embeddings for Router to Remove OpenAI dependency #62

Open madison-evans opened 1 month ago

madison-evans commented 1 month ago

As the title suggests, I want to be able to utilize an open source embedding model from huggingface such that I don't require openAI or any other api driven embedding model for the actual routing done in mf routing. I'm able to dig into the source code in the routellm package and accomplish this by modifying the matrix_factorization.model.py file as well as controller.py and routers.py, but it would be great if this option was already available