How to use the groq models?

lm-sys / RouteLLM

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality!

Apache License 2.0

3.3k stars 250 forks source link

How to use the groq models? #41

Closed Harshpatil317 closed 3 months ago

iojw commented 3 months ago

Hi there, we support any OpenAI compatible endpoint, so you should be able to use it with Groq after setting your API key and updating the base URL. See https://console.groq.com/docs/openai

iojw commented 3 months ago

Closing this for now, let me know if you have other questions!