OpenAI Integration

[ejmockler]

Background

OpenAI's large language models are useful here for moderation, topic classification, amongst other features.

Tasks

• [ ] Implement AI endpoints
  • [ ] Rate-limit, require auth & enforce origin for public API
  • [ ] Determine prompt strategy; multiple agents & factored logic
  • [ ] Moderate endpoint
    • [ ] Shortid & topic generation
    • [ ] Risk classification
  • [ ] Suggest endpoint
  • [ ] Subject line rewording

[krrishdholakia]

Hey @ejmockler, I'm the maintainer of LiteLLM, working on an open-source openai-proxy server to call 100+ llm models.

We already support load balancing + auth for your openai endpoint.

Curious what's missing for us to be helpful today - https://docs.litellm.ai/docs/simple_proxy