pipecat-ai / pipecat

Open Source framework for voice and multimodal conversational AI
BSD 2-Clause "Simplified" License
3.47k stars 341 forks source link

Facing `Unable to connect to Deepgram` issue in given deepgram example #651

Closed adayush closed 1 month ago

adayush commented 1 month ago

Hi,

I was facing issues with Deepgram connection in my project after upgrading from 0.0.41 to 0.0.47 so I tried a deepgram example given in the project but still faced the same issue. I also faced the same issue when trying to upgrade to 0.0.43 a few weeks back when it was the latest but gave up due to this issue. Please help me fix this, I'm tired of trying to make this work.

[Steps I took]

  1. Downloaded 07c-interruptible-deepgram.py and runner.py
  2. Ran python 07c-interruptible-deepgram.py

[System] pipecat-ai 0.0.47 deepgram-sdk 3.7.5 MacOS Sonoma 16GB

[Logs] (I had enabled deepgram debug logs so the extra logs are because of that. Hope it would be useful)

❯ python 07c-interruptible-deepgram.py
2024-10-24 18:23:21.596 | INFO     | pipecat.audio.vad.vad_analyzer:set_params:60 - Setting VAD params to: confidence=0.7 start_secs=0.2 stop_secs=0.8 min_volume=0.6
2024-10-24 18:23:21.596 | DEBUG    | pipecat.audio.vad.silero:__init__:114 - Loading Silero VAD model...
2024-10-24 18:23:21.689 | DEBUG    | pipecat.audio.vad.silero:__init__:136 - Loaded Silero VAD
Version.v ENTER
version: 1
path: deepgram.clients.listen.v1.websocket.async_client
class_name: AsyncListenWebSocketClient
Version.v succeeded
Version.v LEAVE
event subscribed: Results
2024-10-24 18:23:21.701 | DEBUG    | pipecat.processors.frame_processor:link:134 - Linking PipelineSource#0 -> DailyInputTransport#0
2024-10-24 18:23:21.701 | DEBUG    | pipecat.processors.frame_processor:link:134 - Linking DailyInputTransport#0 -> DeepgramSTTService#0
2024-10-24 18:23:21.701 | DEBUG    | pipecat.processors.frame_processor:link:134 - Linking DeepgramSTTService#0 -> OpenAIUserContextAggregator#0
2024-10-24 18:23:21.701 | DEBUG    | pipecat.processors.frame_processor:link:134 - Linking OpenAIUserContextAggregator#0 -> OpenAILLMService#0
2024-10-24 18:23:21.701 | DEBUG    | pipecat.processors.frame_processor:link:134 - Linking OpenAILLMService#0 -> DeepgramTTSService#0
2024-10-24 18:23:21.701 | DEBUG    | pipecat.processors.frame_processor:link:134 - Linking DeepgramTTSService#0 -> DailyOutputTransport#0
2024-10-24 18:23:21.701 | DEBUG    | pipecat.processors.frame_processor:link:134 - Linking DailyOutputTransport#0 -> OpenAIAssistantContextAggregator#0
2024-10-24 18:23:21.701 | DEBUG    | pipecat.processors.frame_processor:link:134 - Linking OpenAIAssistantContextAggregator#0 -> PipelineSink#0
2024-10-24 18:23:21.701 | DEBUG    | pipecat.processors.frame_processor:link:134 - Linking Source#0 -> Pipeline#0
2024-10-24 18:23:21.701 | DEBUG    | pipecat.processors.frame_processor:link:134 - Linking Pipeline#0 -> Sink#0
2024-10-24 18:23:21.701 | DEBUG    | pipecat.pipeline.runner:run:27 - Runner PipelineRunner#0 started running PipelineTask#0
AsyncListenWebSocketClient.start ENTER
options: {'channels': 1, 'encoding': 'linear16', 'interim_results': True, 'language': <Language.EN: 'en'>, 'model': 'nova-2-general', 'punctuate': True, 'profanity_filter': True, 'sample_rate': 16000, 'smart_format': True, 'vad_events': False}
addons: None
headers: None
members: None
kwargs: {}
AbstractAsyncWebSocketClient.start ENTER
addons: None
headers: None
kwargs: {}
combined_options: {'channels': 1, 'encoding': 'linear16', 'interim_results': True, 'language': <Language.EN: 'en'>, 'model': 'nova-2-general', 'punctuate': True, 'profanity_filter': True, 'sample_rate': 16000, 'smart_format': True, 'vad_events': False}
combined_headers: {'Accept': 'application/json', 'Authorization': 'Token **********', 'User-Agent': '@deepgram/sdk/v3.7.5 python/10.14'}
WebSocketException in AbstractAsyncWebSocketClient.start: server rejected WebSocket connection: HTTP 400
AbstractAsyncWebSocketClient.start LEAVE
AsyncListenWebSocketClient.start failed
AsyncListenWebSocketClient.start LEAVE
2024-10-24 18:23:23.283 | ERROR    | pipecat.services.deepgram:_connect:196 - DeepgramSTTService#0: Unable to connect to Deepgram
2024-10-24 18:23:23.283 | INFO     | pipecat.transports.services.daily:join:301 - Joining https://{domain}.daily.co/audiocall
2024-10-24 18:23:25.927 | INFO     | pipecat.transports.services.daily:join:320 - Joined https://{domain}.daily.co/audiocall
^C2024-10-24 18:23:30.186 | WARNING  | pipecat.pipeline.runner:_sig_handler:51 - Interruption detected. Canceling runner PipelineRunner#0
2024-10-24 18:23:30.186 | DEBUG    | pipecat.pipeline.runner:cancel:38 - Canceling runner PipelineRunner#0
2024-10-24 18:23:30.186 | DEBUG    | pipecat.pipeline.task:cancel:118 - Canceling pipeline task PipelineTask#0
2024-10-24 18:23:30.187 | INFO     | pipecat.transports.services.daily:leave:408 - Leaving https://{domain}.daily.co/audiocall
2024-10-24 18:23:30.203 | INFO     | pipecat.transports.services.daily:leave:417 - Left https://{domain}.daily.co/audiocall
AsyncListenWebSocketClient.finish ENTER
cancelling tasks...
AbstractAsyncWebSocketClient.finish ENTER
closing socket...
clean up socket...
cancelling tasks...
before running thread: MainThread
before running thread: asyncio_0
number of active threads: 2
threads joined
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
finish succeeded
before running thread: MainThread
before running thread: asyncio_0
number of active threads: 2
threads joined
after running thread: MainThread
after running thread: asyncio_0
number of active threads: 2
finish succeeded
2024-10-24 18:23:30.204 | INFO     | pipecat.services.deepgram:_disconnect:201 - DeepgramSTTService#0: Disconnected from Deepgram
2024-10-24 18:23:30.205 | DEBUG    | pipecat.pipeline.runner:run:31 - Runner PipelineRunner#0 finished running PipelineTask#0
adayush commented 1 month ago

Looks like it was an issue with the dependencies not being upgraded with pipecat. Also had to initialise Deepgram like this to make it work:

    client = DeepgramSTTService(
          api_key=os.getenv("DEEPGRAM_API_KEY"),
          live_options=LiveOptions(
              model="nova-2",
              language=lang_code,
              sample_rate=16000,
              encoding="linear16",
          ),
      )