GladiaSTTService: Potential Bug in WebSocket Handling

[dominikroblek]

Occasionally, I encounter this error related to GladiaSTTService. Could this be a bug in the GladiaSTTService, the Gladia library, or my code?

2024-09-18 09:30:48.087 | ERROR    | pipecat.processors.frame_processor:push_frame:203 - Uncaught exception in DailyInputTransport#0: sent 1000 (OK); then received 1000 (OK)
Traceback (most recent call last):

  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code

  File "/Users/magnacarta/p/sunray/src/sunray/server.py", line 39, in <module>
    app.run(main)
    │   │   └ <function main at 0x101cae200>
    │   └ <function run at 0x101e68ea0>
    └ <module 'absl.app' from '/Users/magnacarta/p/sunray/.venv/lib/python3.12/site-packages/absl/app.py'>

  File "/Users/magnacarta/p/sunray/.venv/lib/python3.12/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
    │         │     └ ['/Users/magnacarta/p/sunray/src/sunray/server.py']
    │         └ <function main at 0x101cae200>
    └ <function _run_main at 0x101e68d60>
  File "/Users/magnacarta/p/sunray/.venv/lib/python3.12/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
    │   │    │    └ ['/Users/magnacarta/p/sunray/src/sunray/server.py']
    │   │    └ <function main at 0x101cae200>
    │   └ <built-in function exit>
    └ <module 'sys' (built-in)>

  File "/Users/magnacarta/p/sunray/src/sunray/server.py", line 35, in main
    asyncio.run(_run_bot())
    │       │   └ <function _run_bot at 0x13dfc3600>
    │       └ <function run at 0x1009f74c0>
    └ <module 'asyncio' from '/opt/homebrew/Cellar/python@3.12/3.12.6/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyn...

  File "/opt/homebrew/Cellar/python@3.12/3.12.6/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           │      │   └ <coroutine object _run_bot at 0x175cf58c0>
           │      └ <function Runner.run at 0x100a73920>
           └ <asyncio.runners.Runner object at 0x174ff8950>

  File "/opt/homebrew/Cellar/python@3.12/3.12.6/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           │    │     │                  └ <Task pending name='Task-1' coro=<_run_bot() running at /Users/magnacarta/p/sunray/src/sunray/server.py:25> wait_for=<_Gathering...
           │    │     └ <function BaseEventLoop.run_until_complete at 0x100a71440>
           │    └ <_UnixSelectorEventLoop running=True closed=False debug=False>
           └ <asyncio.runners.Runner object at 0x174ff8950>

  File "/opt/homebrew/Cellar/python@3.12/3.12.6/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete
    self.run_forever()
    │    └ <function BaseEventLoop.run_forever at 0x100a713a0>
    └ <_UnixSelectorEventLoop running=True closed=False debug=False>

  File "/opt/homebrew/Cellar/python@3.12/3.12.6/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/base_events.py", line 641, in run_forever
    self._run_once()
    │    └ <function BaseEventLoop._run_once at 0x100a731a0>
    └ <_UnixSelectorEventLoop running=True closed=False debug=False>

  File "/opt/homebrew/Cellar/python@3.12/3.12.6/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/base_events.py", line 1986, in _run_once
    handle._run()
    │      └ <function Handle._run at 0x1009c45e0>
    └ <Handle Task.task_wakeup(<Future finished result=None>)>

  File "/opt/homebrew/Cellar/python@3.12/3.12.6/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
    │    │            │    │           │    └ <member '_args' of 'Handle' objects>
    │    │            │    │           └ <Handle Task.task_wakeup(<Future finished result=None>)>
    │    │            │    └ <member '_callback' of 'Handle' objects>
    │    │            └ <Handle Task.task_wakeup(<Future finished result=None>)>
    │    └ <member '_context' of 'Handle' objects>
    └ <Handle Task.task_wakeup(<Future finished result=None>)>

  File "/Users/magnacarta/p/sunray/.venv/lib/python3.12/site-packages/pipecat/transports/base_input.py", line 134, in _push_frame_task_handler
    await self.push_frame(frame, direction)
          │    │          │      └ <FrameDirection.DOWNSTREAM: 1>
          │    │          └ AudioRawFrame(id=76489, name='AudioRawFrame#54495', audio=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\...
          │    └ <function FrameProcessor.push_frame at 0x139a3b740>
          └ <pipecat.transports.services.daily.DailyInputTransport object at 0x175f18a70>
> File "/Users/magnacarta/p/sunray/.venv/lib/python3.12/site-packages/pipecat/processors/frame_processor.py", line 198, in push_frame
    await self._next.process_frame(frame, direction)
          │    │     │             │      └ <FrameDirection.DOWNSTREAM: 1>
          │    │     │             └ AudioRawFrame(id=76489, name='AudioRawFrame#54495', audio=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\...
          │    │     └ <function GladiaSTTService.process_frame at 0x13cc2f920>
          │    └ <pipecat.services.gladia.GladiaSTTService object at 0x174e1a660>
          └ <pipecat.transports.services.daily.DailyInputTransport object at 0x175f18a70>
  File "/Users/magnacarta/p/sunray/.venv/lib/python3.12/site-packages/pipecat/services/gladia.py", line 66, in process_frame
    await self._send_audio(frame)
          │    │           └ AudioRawFrame(id=76489, name='AudioRawFrame#54495', audio=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\...
          │    └ <function GladiaSTTService._send_audio at 0x13cc2fba0>
          └ <pipecat.services.gladia.GladiaSTTService object at 0x174e1a660>
  File "/Users/magnacarta/p/sunray/.venv/lib/python3.12/site-packages/pipecat/services/gladia.py", line 99, in _send_audio
    await self._websocket.send(json.dumps(message))
          │    │          │    │    │     └ {'frames': 'AAAAAAAAAAAAAAAAAAD//wAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAD//wAAAAAAAAAAAAAAAAAAAQAAAAAAAQAAAAAAAAAAAAAAAAABAAAAA...
          │    │          │    │    └ <function dumps at 0x100b1c680>
          │    │          │    └ <module 'json' from '/opt/homebrew/Cellar/python@3.12/3.12.6/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/__...
          │    │          └ <function WebSocketCommonProtocol.send at 0x13c9cfe20>
          │    └ <websockets.legacy.client.WebSocketClientProtocol object at 0x174d53560>
          └ <pipecat.services.gladia.GladiaSTTService object at 0x174e1a660>
  File "/Users/magnacarta/p/sunray/.venv/lib/python3.12/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
          │    └ <function WebSocketCommonProtocol.ensure_open at 0x13c9f4220>
          └ <websockets.legacy.client.WebSocketClientProtocol object at 0x174d53560>
  File "/Users/magnacarta/p/sunray/.venv/lib/python3.12/site-packages/websockets/legacy/protocol.py", line 948, in ensure_open
    raise self.connection_closed_exc()
          │    └ <function WebSocketCommonProtocol.connection_closed_exc at 0x13c9f4180>
          └ <websockets.legacy.client.WebSocketClientProtocol object at 0x174d53560>

websockets.exceptions.ConnectionClosedOK: sent 1000 (OK); then received 1000 (OK)

[nmaswood]

@amacapri I have been getting this error every once and while:

2024-09-25 14:44:38.960 | ERROR    | pipecat.processors.frame_processor:push_frame:203 - Uncaught exception in DailyInputTransport#0: 'GladiaSTTService' object has no attribute '_websocket'
Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/Users/nasrmaswood/code/voice-demo/python/pipecat_demo/pipecat_demo/main.py", line 124, in main
    asyncio.run(runner())
    │       │   └ <function runner at 0x1032702c0>
    │       └ <function run at 0x103ce3600>
    └ <module 'asyncio' from '/usr/local/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/async...

  File "/usr/local/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           │      │   └ <coroutine object runner at 0x1212028c0>
           │      └ <function Runner.run at 0x103cef7e0>
           └ <asyncio.runners.Runner object at 0x1031afd10>

  File "/usr/local/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           │    │     │                  └ <Task pending name='Task-1' coro=<runner() running at /Users/nasrmaswood/code/voice-demo/python/pipecat_demo/pipecat_demo/mai...
           │    │     └ <function BaseEventLoop.run_until_complete at 0x103ced440>
           │    └ <_UnixSelectorEventLoop running=True closed=False debug=False>
           └ <asyncio.runners.Runner object at 0x1031afd10>

  File "/usr/local/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 641, in run_until_complete
    self.run_forever()
    │    └ <function BaseEventLoop.run_forever at 0x103ced3a0>
    └ <_UnixSelectorEventLoop running=True closed=False debug=False>

  File "/usr/local/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 608, in run_forever
    self._run_once()
    │    └ <function BaseEventLoop._run_once at 0x103cef1a0>
    └ <_UnixSelectorEventLoop running=True closed=False debug=False>

  File "/usr/local/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 1936, in _run_once
    handle._run()
    │      └ <function Handle._run at 0x103c35260>
    └ <Handle Task.task_wakeup(<Task finishe...> result=None>)>

  File "/usr/local/Cellar/python@3.11/3.11.9_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/events.py", line 84, in _run
    self._context.run(self._callback, *self._args)
    │    │            │    │           │    └ <member '_args' of 'Handle' objects>
    │    │            │    │           └ <Handle Task.task_wakeup(<Task finishe...> result=None>)>
    │    │            │    └ <member '_callback' of 'Handle' objects>
    │    │            └ <Handle Task.task_wakeup(<Task finishe...> result=None>)>
    │    └ <member '_context' of 'Handle' objects>
    └ <Handle Task.task_wakeup(<Task finishe...> result=None>)>

  File "/Users/nasrmaswood/Library/Caches/pypoetry/virtualenvs/pipecat-demo-_ugJuq0G-py3.11/lib/python3.11/site-packages/pipecat/pipeline/task.py", line 89, in cancel
    await self._source.push_frame(CancelFrame())
          │    │       │          └ <class 'pipecat.frames.frames.CancelFrame'>
          │    │       └ <function FrameProcessor.push_frame at 0x11a371260>
          │    └ <pipecat.pipeline.task.Source object at 0x12139ffd0>
          └ <pipecat.pipeline.task.PipelineTask object at 0x11d0ae6d0>
  File "/Users/nasrmaswood/Library/Caches/pypoetry/virtualenvs/pipecat-demo-_ugJuq0G-py3.11/lib/python3.11/site-packages/pipecat/processors/frame_processor.py", line 198, in push_frame
    await self._next.process_frame(frame, direction)
          │    │     │             │      └ <FrameDirection.DOWNSTREAM: 1>
          │    │     │             └ CancelFrame(id=2755, name='CancelFrame#0')
          │    │     └ <function Pipeline.process_frame at 0x11a3716c0>
          │    └ <pipecat.pipeline.pipeline.Pipeline object at 0x12139ef50>
          └ <pipecat.pipeline.task.Source object at 0x12139ffd0>
  File "/Users/nasrmaswood/Library/Caches/pypoetry/virtualenvs/pipecat-demo-_ugJuq0G-py3.11/lib/python3.11/site-packages/pipecat/pipeline/pipeline.py", line 83, in process_frame
    await self._source.process_frame(frame, FrameDirection.DOWNSTREAM)
          │    │       │             │      │              └ <FrameDirection.DOWNSTREAM: 1>
          │    │       │             │      └ <enum 'FrameDirection'>
          │    │       │             └ CancelFrame(id=2755, name='CancelFrame#0')
          │    │       └ <function PipelineSource.process_frame at 0x11a2fbce0>
          │    └ <pipecat.pipeline.pipeline.PipelineSource object at 0x12139f010>
          └ <pipecat.pipeline.pipeline.Pipeline object at 0x12139ef50>
  File "/Users/nasrmaswood/Library/Caches/pypoetry/virtualenvs/pipecat-demo-_ugJuq0G-py3.11/lib/python3.11/site-packages/pipecat/pipeline/pipeline.py", line 27, in process_frame
    await self.push_frame(frame, direction)
          │    │          │      └ <FrameDirection.DOWNSTREAM: 1>
          │    │          └ CancelFrame(id=2755, name='CancelFrame#0')
          │    └ <function FrameProcessor.push_frame at 0x11a371260>
          └ <pipecat.pipeline.pipeline.PipelineSource object at 0x12139f010>
  File "/Users/nasrmaswood/Library/Caches/pypoetry/virtualenvs/pipecat-demo-_ugJuq0G-py3.11/lib/python3.11/site-packages/pipecat/processors/frame_processor.py", line 198, in push_frame
    await self._next.process_frame(frame, direction)
          │    │     │             │      └ <FrameDirection.DOWNSTREAM: 1>
          │    │     │             └ CancelFrame(id=2755, name='CancelFrame#0')
          │    │     └ <function DailyInputTransport.process_frame at 0x11b2cd1c0>
          │    └ <pipecat.transports.services.daily.DailyInputTransport object at 0x121396ed0>
          └ <pipecat.pipeline.pipeline.PipelineSource object at 0x12139f010>
  File "/Users/nasrmaswood/Library/Caches/pypoetry/virtualenvs/pipecat-demo-_ugJuq0G-py3.11/lib/python3.11/site-packages/pipecat/transports/services/daily.py", line 611, in process_frame
    await super().process_frame(frame, direction)
                                │      └ <FrameDirection.DOWNSTREAM: 1>
                                └ CancelFrame(id=2755, name='CancelFrame#0')
  File "/Users/nasrmaswood/Library/Caches/pypoetry/virtualenvs/pipecat-demo-_ugJuq0G-py3.11/lib/python3.11/site-packages/pipecat/transports/base_input.py", line 87, in process_frame
    await self.push_frame(frame, direction)
          │    │          │      └ <FrameDirection.DOWNSTREAM: 1>
          │    │          └ CancelFrame(id=2755, name='CancelFrame#0')
          │    └ <function FrameProcessor.push_frame at 0x11a371260>
          └ <pipecat.transports.services.daily.DailyInputTransport object at 0x121396ed0>
> File "/Users/nasrmaswood/Library/Caches/pypoetry/virtualenvs/pipecat-demo-_ugJuq0G-py3.11/lib/python3.11/site-packages/pipecat/processors/frame_processor.py", line 198, in push_frame
    await self._next.process_frame(frame, direction)
          │    │     │             │      └ <FrameDirection.DOWNSTREAM: 1>
          │    │     │             └ CancelFrame(id=2755, name='CancelFrame#0')
          │    │     └ <function GladiaSTTService.process_frame at 0x11b2120c0>
          │    └ <pipecat.services.gladia.GladiaSTTService object at 0x11b3cb210>
          └ <pipecat.transports.services.daily.DailyInputTransport object at 0x121396ed0>
  File "/Users/nasrmaswood/Library/Caches/pypoetry/virtualenvs/pipecat-demo-_ugJuq0G-py3.11/lib/python3.11/site-packages/pipecat/services/gladia.py", line 61, in process_frame
    await super().process_frame(frame, direction)
                                │      └ <FrameDirection.DOWNSTREAM: 1>
                                └ CancelFrame(id=2755, name='CancelFrame#0')
  File "/Users/nasrmaswood/Library/Caches/pypoetry/virtualenvs/pipecat-demo-_ugJuq0G-py3.11/lib/python3.11/site-packages/pipecat/services/ai_services.py", line 87, in process_frame
    await self.cancel(frame)
          │    │      └ CancelFrame(id=2755, name='CancelFrame#0')
          │    └ <function GladiaSTTService.cancel at 0x11b23fec0>
          └ <pipecat.services.gladia.GladiaSTTService object at 0x11b3cb210>
  File "/Users/nasrmaswood/Library/Caches/pypoetry/virtualenvs/pipecat-demo-_ugJuq0G-py3.11/lib/python3.11/site-packages/pipecat/services/gladia.py", line 82, in cancel
    await self._websocket.close()
          └ <pipecat.services.gladia.GladiaSTTService object at 0x11b3cb210>

[nmaswood]

It looks the errors are different but related. I am going to dig a little bit