[Feature]: Support new openai `/v1/realtime` websocket endpoints

BerriAI / litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

https://docs.litellm.ai/docs/

Other

13.92k stars 1.64k forks source link

[Feature]: Support new openai `/v1/realtime` websocket endpoints #6045

Closed krrishdholakia closed 1 month ago

krrishdholakia commented 1 month ago

The Feature

Used by OpenAI for their new gpt-4o audio model - gpt-4o-realtime-preview-2024-10-01

Motivation, pitch

OpenAI-compatibility

Twitter / LinkedIn details

No response

krrishdholakia commented 1 month ago

Able to get azure endpoint working.

OpenAI giving 401/403 errors.

krrishdholakia commented 1 month ago

Initial working commit made.

Todo:

[x] Expose abstraction litellm.arealtime(..)
[x] Expose on Router router.arealtime(..) -> enables eventual loadbalancing b/w openai + azure
[x] Refactor proxy /v1/realtime endpoint to use router