p-i- / openai-realtime-py

Python implementation of OpenAI's realtime API
MIT License
44 stars 5 forks source link

Suggestion: turn this into a Python-based agnostic relay server #2

Open mor10 opened 1 month ago

mor10 commented 1 month ago

Like you said in your readme, Python is not ideal for doing the actual streaming of content through WebSockets, but is ideal as pass-through / proxy / middleware managing the security layer of a front-end app.

Have you considered building this into a straightforward plug-and-play implementation of a Python back-end that handles keys and exchanges that any front-end app can pull from easily? So rather than the Python app doing all the WebSockets work, the python apps works as an intermediary and leaves the front-end app to do whatever it wants to do.

It's an unfinished thought. Do with it as you please.

p-i- commented 1 month ago

I'm working on a C++ cross-platform implementation that may be similar to what you describe. The idea being that the C++ handles the realtime audio and websocketry while exposing a generic interface to allow e.g. tool-calling and other such stuff. I might end up with a bidirectional websocket relay that just doesn't contain any of the audio interactions.

Patrickeik commented 1 month ago

when i run the mainfile: i get just lines of code scrolling over my screen live

p-i- commented 1 month ago

Try running the legacy file. That's the only one that works cleanly. Does that work?