search

pipecat-ai / pipecat

Open Source framework for voice and multimodal conversational AI
BSD 2-Clause "Simplified" License
3.12k stars 272 forks source link

Daily join timeout / Deepgram STT timeout #457

Closed bigahega closed 1 week ago

bigahega commented 2 weeks ago

Sometimes daily sdk raises a timeout warning while joining the room. In this case if daily transcriptions are enabled, nothing gets transcribed.

A similar issue happens when deepgram TTS is used explicitly without enabling the daily transcription. If joining the room times out, deepgram sdk raises timeout errors and does not transcribe anything further even if daily later on joins the room successfully.

I'll add logs in short.

bigahega commented 2 weeks ago

First daily sdk prints the following log:

ERROR | pipecat.transports.services.daily:join:316 - Time out joining https://******.daily.co/uuqM706DW6gGDtPRw72P

and right after this log, deepgram sdk prints the following:

ConnectionClosed in AsyncListenWebSocketClient._listening with code 1011: Deepgram did not receive audio data or a text message within the timeout window. See https://dpgr.am/net0001 async_client.py :403 2024-09-13 12:33:52,825 ConnectionClosed in AsyncListenWebSocketClient._listening with code 1011: Deepgram did not receive audio data or a text message within the timeout window. See https://dpgr.am/net0001

this is for the case daily.co + deepgram stt

For the daily.co and its own transcription case, first log is the same; daily errors out saying joining the room timed out and that's it. In a few seconds it connects but it does not emit any transcription.

golbin commented 2 weeks ago

Maybe, I have the same issus.

2024-09-14 15:46:20.742 | DEBUG    | pipecat.processors.frame_processor:link:169 - Linking Source#0 -> Pipeline#0
2024-09-14 15:46:20.742 | DEBUG    | pipecat.pipeline.runner:run:28 - Runner PipelineRunner#0 started running PipelineTask#0
2024-09-14 15:46:20.742 | INFO     | pipecat.transports.services.daily:join:281 - Joining https://---.daily.co/---
2024-09-14 15:46:22.078 | INFO     | pipecat.transports.services.daily:join:303 - Joined https://---.daily.co/---
WebSocketException in AsyncListenWebSocketClient.start:
2024-09-14 15:46:32.084 | ERROR    | pipecat.services.deepgram:start:149 - DeepgramSTTService#0: Unable to connect to Deepgram
bigahega commented 2 weeks ago

I have seen the same deepgram log this morning.

"message":"2024-09-16 07:45:32.345 | ERROR | pipecat.services.deepgram:_connect:171 - DeepgramSTTService#0: Unable to connect to Deepgram\n"

Is it maybe possible to add a retry mechanism? Daily.co retries joining automatically but if deepgram fails, it's a dead end.

bigahega commented 1 week ago

After much digging around, I found out this was due to some of my other code blocking the asyncio event loop whilst joining the room.