Closed NalaGinrut closed 3 months ago
I'd like to use my own local LLM, or other cloud LLM service, say Groq. And I think they're OpenAI API compatible, so it's better to support customizable LLM API gateway that people may add their prefer LLM.
I'd like to use my own local LLM, or other cloud LLM service, say Groq. And I think they're OpenAI API compatible, so it's better to support customizable LLM API gateway that people may add their prefer LLM.