2024-09-19 01:58:32,080 - WARNING livekit.agents - Running <Task pending name='Task-33' coro=<_stream_synthesis_task() running at ~/env/lib/python3.12/site-packages/livekit/agents/utils/log.py:16> cb=[AgentOutput._synthesize_task.<locals>.<lambda>() at ~/env/lib/python3.12/site-packages/livekit/agents/voice_assistant/agent_output.py:164, _wait.<locals>._on_completion() at /usr/local/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/tasks.py:534]> took too long: 15.24 seconds {"pid": 94165, "job_id": "AJ_bGsA4AR6ykdb"}
2024-09-19 01:58:32,087 - ERROR livekit.agents - Error in _read_ipc_task
Traceback (most recent call last):
File "~/env/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/env/lib/python3.12/site-packages/livekit/agents/ipc/proc_main.py", line 195, in _read_ipc_task
no_msg_timeout.reset()
File "~/env/lib/python3.12/site-packages/livekit/agents/utils/aio/sleep.py", line 34, in reset
raise SleepFinished
livekit.agents.utils.aio.sleep.SleepFinished {"pid": 94165, "job_id": "AJ_bGsA4AR6ykdb"}
worker process is not responding.. worker crashed?
2024-09-19 01:58:32,094 - WARNING livekit.plugins.deepgram - deepgram connection failed, retrying in 0s
Traceback (most recent call last):
File "~/env/lib/python3.12/site-packages/livekit/plugins/deepgram/stt.py", line 334, in _run_ws
await asyncio.gather(*tasks)
File "~/env/lib/python3.12/site-packages/livekit/plugins/deepgram/stt.py", line 316, in recv_task
raise Exception("deepgram connection closed unexpectedly")
Exception: deepgram connection closed unexpectedly
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "~/env/lib/python3.12/site-packages/livekit/plugins/deepgram/stt.py", line 243, in _run
await self._run_ws(ws)
File "~/env/lib/python3.12/site-packages/livekit/plugins/deepgram/stt.py", line 336, in _run_ws
await utils.aio.gracefully_cancel(*tasks)
File "~/env/lib/python3.12/site-packages/livekit/agents/utils/aio/__init__.py", line 16, in gracefully_cancel
await asyncio.gather(*futures)
File "~/env/lib/python3.12/site-packages/livekit/plugins/deepgram/stt.py", line 297, in send_task
await ws.send_bytes(frame.data.tobytes())
File "~/env/lib/python3.12/site-packages/aiohttp/client_ws.py", line 230, in send_bytes
await self._writer.send(data, binary=True, compress=compress)
File "~/env/lib/python3.12/site-packages/aiohttp/http_websocket.py", line 724, in send
await self._send_frame(message, WSMsgType.BINARY, compress)
File "~/env/lib/python3.12/site-packages/aiohttp/http_websocket.py", line 626, in _send_frame
raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport {"pid": 94165, "job_id": "AJ_bGsA4AR6ykdb"}
Process job_proc:
Traceback (most recent call last):
File "/usr/local/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/local/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "~/env/lib/python3.12/site-packages/livekit/agents/ipc/proc_main.py", line 272, in main
loop.run_until_complete(main_task)
File "/usr/local/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "~/env/lib/python3.12/site-packages/livekit/agents/ipc/proc_main.py", line 233, in _async_main
await utils.aio.gracefully_cancel(read_task, health_check_task)
File "~/env/lib/python3.12/site-packages/livekit/agents/utils/aio/__init__.py", line 16, in gracefully_cancel
await asyncio.gather(*futures)
File "~/env/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/env/lib/python3.12/site-packages/livekit/agents/ipc/proc_main.py", line 195, in _read_ipc_task
no_msg_timeout.reset()
File "~/env/lib/python3.12/site-packages/livekit/agents/utils/aio/sleep.py", line 34, in reset
raise SleepFinished
livekit.agents.utils.aio.sleep.SleepFinished
AphinityApp
commented
1 week ago
How to reproduce
Not able to reproduce consistently, but it happens occasionally with the example code provided in https://github.com/livekit/agents/blob/main/examples/voice-assistant/minimal_assistant.py
Might be able to trigger it locally if you can block deepgram through the OS.
Setup
Python 3.12.4 on macOS 14.6.1 M1 ARM64
requirements.txt.lockLogs