The worker periodically disconnects

[wezerr]

The worker periodically disconnects. There are no loads on the CPU and RAM. No correlation has been found with the number of active rooms. Could you please advise what might be causing this?

Error code:

source_traceback: Object created at (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.11/multiprocessing/spawn.py", line 122, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/usr/local/lib/python3.11/multiprocessing/spawn.py", line 135, in _main
    return self._bootstrap(parent_sentinel)
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/appuser/.local/lib/python3.11/site-packages/livekit/agents/cli/cli.py", line 242, in run_worker
    loop.run_until_complete(main_task)
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 640, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 607, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1914, in _run_once
    handle._run()
  File "/home/appuser/.local/lib/python3.11/site-packages/livekit/agents/aio/debug.py", line 13, in instrumented
    val = _run(self)
  File "/usr/local/lib/python3.11/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/appuser/.local/lib/python3.11/site-packages/livekit/agents/apipe.py", line 37, in _put_msg
    _ = asyncio.ensure_future(self._read_ch.send(msg))
  File "/usr/local/lib/python3.11/asyncio/tasks.py", line 659, in ensure_future
    return _ensure_future(coro_or_future, loop=loop)
  File "/usr/local/lib/python3.11/asyncio/tasks.py", line 680, in _ensure_future
    return loop.create_task(coro_or_future)

Additional Information:

Python Version: 3.11 Operating System: Debian Package Version:

livekit-agents~=0.7.2
livekit
python-dotenv
boto3
openai
livekit-plugins-openai~=0.6
livekit-plugins-deepgram~=0.5
livekit-plugins-elevenlabs~=0.6
livekit-plugins-silero~=0.5

[theomonnom]

I only see the error traceback, is there any other error messages?

[wezerr]

We are receiving several thousand of such errors per second, which makes it difficult to track any other potential error messages. Unfortunately, we do not have any additional error messages at the moment.

If there's any other specific information you need or any steps you would like us to take to gather more details, please let us know.

Thank you for your assistance!

[theomonnom]

It's unclear what is happening. Can you give livekit-agents v0.8.0 a try? We made quite a lot of changes in this version, including how IPC works

[wezerr]

If there are no other options available, we will give livekit-agents v0.8.0 a try. Thanks again!

[theomonnom]

Yes, livekit/agents/apipe.py has been replaced by ProcChannel in v0.8.0. So this was maybe an old issue?

[wezerr]

I haven't had a chance to test it on the new version yet, maybe after the weekend

[wezerr]

Hi everyone, I wanted to update you that we've tested livekit-agents v0.8.0, and it seems to have resolved the issue. Everything is working fine now, so we can close this issue. Thank you for your help!