giorgosn
commented
2 days ago Are you talking about this error? `
WARNING ⚠️ WebSocket connection lost. Reconnecting... logging.py:74
INFO 🎤 ⬆️ - Out input_audio_buffer.append logging.py:6
1
ERROR WebSocket connection closed unexpectedly. main.py:339 ╭──────────────────────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────────────────────╮
│ C:\Users\Npizza\Desktop\AI experiments\AI-realtime.venv\Lib\site-packages\websockets\legacy\protocol.py:953 in transfer_data │
│ │
│ 950 │ │ """ │
│ 951 │ │ try: │
│ 952 │ │ │ while True: │
│ ❱ 953 │ │ │ │ message = await self.read_message() │
│ 954 │ │ │ │ │
│ 955 │ │ │ │ # Exit the loop when receiving a close frame. │
│ 956 │ │ │ │ if message is None: │
│ │
│ C:\Users\Npizza\Desktop\AI experiments\AI-realtime.venv\Lib\site-packages\websockets\legacy\protocol.py:1023 in read_message │
│ │
│ 1020 │ │ Return :obj:None when the closing handshake is started. │
│ 1021 │ │ │
│ 1022 │ │ """ │
│ ❱ 1023 │ │ frame = await self.read_data_frame(max_size=self.max_size) │
│ 1024 │ │ │
│ 1025 │ │ # A close frame was received. │
│ 1026 │ │ if frame is None: │
│ │
│ C:\Users\Npizza\Desktop\AI experiments\AI-realtime.venv\Lib\site-packages\websockets\legacy\protocol.py:1098 in read_data_frame │
│ │
│ 1095 │ │ """ │
│ 1096 │ │ # 6.2. Receiving Data │
│ 1097 │ │ while True: │
│ ❱ 1098 │ │ │ frame = await self.read_frame(max_size) │
│ 1099 │ │ │ │
│ 1100 │ │ │ # 5.5. Control Frames │
│ 1101 │ │ │ if frame.opcode == OP_CLOSE: │
│ │
│ C:\Users\Npizza\Desktop\AI experiments\AI-realtime.venv\Lib\site-packages\websockets\legacy\protocol.py:1155 in read_frame │
│ │
│ 1152 │ │ Read a single frame from the connection. │
│ 1153 │ │ │
│ 1154 │ │ """ │
│ ❱ 1155 │ │ frame = await Frame.read( │
│ 1156 │ │ │ self.reader.readexactly, │
│ 1157 │ │ │ mask=not self.is_client, │
│ 1158 │ │ │ max_size=max_size, │
│ │
│ C:\Users\Npizza\Desktop\AI experiments\AI-realtime.venv\Lib\site-packages\websockets\legacy\framing.py:70 in read │
│ │
│ 67 │ │ """ │
│ 68 │ │ │
│ 69 │ │ # Read the header. │
│ ❱ 70 │ │ data = await reader(2) │
│ 71 │ │ head1, head2 = struct.unpack("!BB", data) │
│ 72 │ │ │
│ 73 │ │ # While not Pythonic, this is marginally faster than calling bool(). │
│ │
│ C:\Users\Npizza\anaconda3\envs\py312\Lib\asyncio\streams.py:750 in readexactly │
│ │
│ 747 │ │ │ if self._eof: │
│ 748 │ │ │ │ incomplete = bytes(self._buffer) │
│ 749 │ │ │ │ self._buffer.clear() │
│ ❱ 750 │ │ │ │ raise exceptions.IncompleteReadError(incomplete, n) │
│ 751 │ │ │ │
│ 752 │ │ │ await self._wait_for_data('readexactly') │
│ 753 │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
IncompleteReadError: 0 bytes read on a total of 2 expected bytes
The above exception was the direct cause of the following exception:
╭──────────────────────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────────────────────╮
│ C:\Users\Npizza\Desktop\AI experiments\AI-realtime\src\realtime_api_async_python\main.py:311 in realtime_api │
│ │
│ 308 │ │ │ │ │ │ │ │ │ │ "audio": base64_audio, │
│ 309 │ │ │ │ │ │ │ │ │ } │
│ 310 │ │ │ │ │ │ │ │ │ log_ws_event("Outgoing", audio_event) │
│ ❱ 311 │ │ │ │ │ │ │ │ │ await websocket.send(json.dumps(audio_event)) │
│ 312 │ │ │ │ │ │ │ │ else: │
│ 313 │ │ │ │ │ │ │ │ │ logger.debug("No audio data to send") │
│ 314 │ │ │ │ │ │ else: │
│ │
│ C:\Users\Npizza\Desktop\AI experiments\AI-realtime\.venv\Lib\site-packages\websockets\legacy\protocol.py:628 in send │
│ │
│ 625 │ │ │ TypeError: If ``message`` doesn't have a supported type. │
│ 626 │ │ │
│ 627 │ │ """ │
│ ❱ 628 │ │ await self.ensure_open() │
│ 629 │ │ │
│ 630 │ │ # While sending a fragmented message, prevent sending other messages │
│ 631 │ │ # until all fragments are sent. │
│ │
│ C:\Users\Npizza\Desktop\AI experiments\AI-realtime\.venv\Lib\site-packages\websockets\legacy\protocol.py:929 in ensure_open │
│ │
│ 926 │ │ │ │ return │
│ 927 │ │ │
│ 928 │ │ if self.state is State.CLOSED: │
│ ❱ 929 │ │ │ raise self.connection_closed_exc() │
│ 930 │ │ │
│ 931 │ │ if self.state is State.CLOSING: │
│ 932 │ │ │ # If we started the closing handshake, wait for its completion to │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ConnectionClosedError: no close frame received or sent`
Issue 2: Audio Buffer Management Failure
Description
The audio buffer occasionally remains empty when attempting to commit audio data, resulting in errors and preventing audio data from being sent to the server.
Steps to Reproduce
Error: Error committing input audio buffer: the buffer is empty.andReceived 'buffer is empty' error, no audio data sent.Expected Behavior
The audio buffer should consistently contain audio data when committing, ensuring that the assistant receives the intended audio input.
Actual Behavior
Sometimes, the audio buffer is empty, leading to errors and no audio data being sent to the server.
Potential Solutions