Open ejmockler opened 10 months ago
Hey @ejmockler, I'm the maintainer of LiteLLM, working on an open-source openai-proxy server to call 100+ llm models.
We already support load balancing + auth for your openai endpoint.
Curious what's missing for us to be helpful today - https://docs.litellm.ai/docs/simple_proxy
Background
OpenAI's large language models are useful here for moderation, topic classification, amongst other features.
Tasks