search

openai / openai-realtime-api-beta

Node.js + JavaScript reference client for the Realtime API (beta)
MIT License
590 stars 127 forks source link

Interruption of assistant in VAD mode not working #54

Open SuperNiceDev opened 1 week ago

SuperNiceDev commented 1 week ago

In VAD mode it seems you cannot interrupt the assistant. It makes a short break, but then keeps on going.

Cazforshort commented 1 week ago

I agree, this seems to be a common issue. It also seems to interrupt itself a lot.

SuperNiceDev commented 1 week ago

Thanks for the confirmation.

Cazforshort commented 1 week ago

I was able to get it working pretty well. Heres how

  1. System constantly processes microphone input at regular intervals (every 0.1 seconds)
  2. Input is converted to PCM16 format and sent to OpenAI's server
  3. Server performs Voice Activity Detection (VAD)

User speaks during AI audio → → Server detects speech ("input_audio_buffer.speech_started" event) → → AudioManager stops current audio playback → System cancels active response → All states reset (isResponseActive, activeResponseId, etc.) → System ready for new interaction