Failing to update instruction and role during active session

openai / openai-realtime-api-beta

Node.js + JavaScript reference client for the Realtime API (beta)

MIT License

715 stars 185 forks source link

Failing to update instruction and role during active session #65

Open emorling opened 3 weeks ago

emorling commented 3 weeks ago

From the documentation it seems like we should be able to update the instructions and the voice during active session?

I am able to run updateSession without any errors.

I can even see the instructions and the voice has changed in this object: console.log('🎯 Final client state:', clientRef.current.sessionConfig);

However, the current client does not use the updated prompt or voice. Do we need to disconnect, and reconnect for changes to take place?

emorling commented 3 weeks ago

By enabling the debug flag when instantiating the RealtimeClient, I was able to see error that I cant switch voice when audio for assistant is active. So I am able to get this working by disconnect() and reconnect(). Is this the only way? I thought updateSession implied we could make changes in flight?