search

sashabaranov / go-openai

OpenAI ChatGPT, GPT-3, GPT-4, DALL·E, Whisper API wrapper for Go
Apache License 2.0
9.29k stars 1.43k forks source link

OpenAI Realtime API support #869

Open bachittle opened 1 month ago

bachittle commented 1 month ago

https://platform.openai.com/docs/guides/realtime

it uses websockets to get and set audio files from gpt-4o-realtime.

anhao commented 1 month ago

+1

WqyJh commented 1 month ago

+1

WqyJh commented 1 month ago

I want to add support for the GPT-4o-realtime model, which relies on WebSocket technology. This necessitates the use of a WebSocket library, but introducing such a dependency conflicts with the zero-dependency philosophy of the existing library, as I previously discussed with @sashabaranov.

image

As a result, I've decided to create a new library dedicated exclusively to GPT-4o-realtime. This library will serve as a complement to go-openai, focusing solely on supporting GPT-4o-realtime functionality.

The new library is called go-openai-realtime. Feel free to check it out!

https://github.com/WqyJh/go-openai-realtime

sashabaranov commented 1 month ago

@WqyJh, thank you for your effort on this! I think websockets are a fair case to introduce a dependency to this library, and I would love to merge your changes if you'll decide to contribute 🙌🏻

bachittle commented 1 month ago

They just released support for audio input and output in the chat completions endpoint, using the gpt-4o-audio-preview model. This could be supported first in the meantime: https://platform.openai.com/docs/guides/audio/quickstart

WqyJh commented 2 weeks ago

They just released support for audio input and output in the chat completions endpoint, using the gpt-4o-audio-preview model. This could be supported first in the meantime: https://platform.openai.com/docs/guides/audio/quickstart

I just added support for gpt-4o-audio-preview. See https://github.com/sashabaranov/go-openai/pull/895

WqyJh commented 2 weeks ago

@WqyJh, thank you for your effort on this! I think websockets are a fair case to introduce a dependency to this library, and I would love to merge your changes if you'll decide to contribute 🙌🏻

I'd like to contribute to the project. Since it contains a lot of code and examples, it will take some time to complete. Mixing all the code together would create a mess, so I suggest organizing all the real-time code into a folder named realtime.