Using realtime API with python client

[someshfengde]

Confirm this is a feature request for the Python library and not the underlying OpenAI API.

• [X] This is a feature request for the Python library

Describe the feature or improvement you're requesting

I had fastAPI server running on my application already. I wanted to make it's user interface voice to voice type. On openai's official documentation only steps for node.js is given. I was wondering if I'd be able to reuse my existing code ( fastapi + openai python ) for voice interface too.

Additional context

Realtime API Docs : https://platform.openai.com/docs/guides/realtime

currently not sure how can I use openai-python for realtime API.

[hayescode]

Microsoft has a python version you can use. Please note that there are connection differences between OpenAI and Azure OpenAI that you'll need to change, but the rest should work for you.

https://github.com/Azure-Samples/aoai-realtime-audio-sdk/tree/main/python

[SergioB-dev]

I'm hoping to see this feature land here as well.

[someshfengde]

If I'm not wrong azure-openai does support openai directly. (From their implmentation it seems like they do)

they've method for getting messages with openai.

https://github.com/Azure-Samples/aoai-realtime-audio-sdk/blob/8105a5c3ab9cc54fe864aa6f8259f72c6829eec7/python/samples/low_level_sample.py#L251

[someshfengde]

Hi @hayescode I did try it was establishing the connection but it requires audio files path as input. If you have any information do you know how can I stream the audio directly to the client? Here what I have to do is save to audio file and then send to openai

[hayescode]

You'll need a frontend to stream and decode the audio chunks. I use Chainlit which implements this differently but it's still python backend, if just you're looking for a quick POC try this:

https://github.com/Chainlit/cookbook/tree/main/realtime-assistant

[someshfengde]

thank you :)