Deepgram module latency issue

[Ajinkya009]

Hi First off, thank you for building and maintaining this amazing package. Our company is delivering real time voice capabilities to millions of customers of large enterprises in India, and want to leverage the Deepgram module in production setting. We use Lua scripting language. We've managed to integrate the module with Freeswitch setup successfully. However, we're noticing a latency of at least 3 seconds between when the user stops speaking and when Freeswitch receives the response from Deepgram servers. Additionally, in the initial phase of the call, we're seeing a number of "discarding empty deepgram transcript" messages on the console. I have attached logs below. The speaker started speaking at 20:31:40 and spoke for approx. 2 seconds. The server sent the transcription around 20:31:46. This means there is a latency of 4 seconds. We would like to reduce the latency well below 1 second. Could you please help us with this.

2024-03-07 20:31:40.443757 99.93% [INFO] mod_deepgram_transcribe.c:154 start transcribing hi interim
2024-03-07 20:31:40.443757 99.93% [DEBUG] dg_transcribe_glue.cpp:336 path: /v1/listen?tier=enhanced&model=general&language=hi&smart_format=true&no_delay=true&punctuate=true&interim_results=true&encoding=linear16&sample_rate=8000
2024-03-07 20:31:40.443757 99.93% [DEBUG] dg_transcribe_glue.cpp:377 (64) no resampling needed for this call
2024-03-07 20:31:40.443757 99.93% [DEBUG] dg_transcribe_glue.cpp:380 (64) fork_data_init
2024-03-07 20:31:40.443757 99.93% [DEBUG] dg_transcribe_glue.cpp:452 connecting now
2024-03-07 20:31:40.443757 99.93% [DEBUG] dg_transcribe_glue.cpp:454 connection in progress
2024-03-07 20:31:40.443757 99.93% [DEBUG] mod_deepgram_transcribe.c:41 Got SWITCH_ABC_TYPE_INIT.
2024-03-07 20:31:40.443757 99.93% [DEBUG] switch_core_media_bug.c:976 Attaching BUG to sofia/tata-profile/09429515176
2024-03-07 20:31:40.443757 99.93% [DEBUG] mod_deepgram_transcribe.c:98 added media bug for dg transcribe
2024-03-07 20:31:40.443757 99.93% [DEBUG] switch_cpp.cpp:86 bound to CUSTOM deepgram_transcribe::transcription
2024-03-07 20:31:40.463756 99.93% [DEBUG] switch_core_io.c:448 Setting BUG Codec PCMU:0
2024-03-07 20:31:41.783759 99.93% [INFO] dg_transcribe_glue.cpp:279 connection successful
2024-03-07 20:31:43.203760 99.93% [DEBUG] dg_transcribe_glue.cpp:305 discarding empty deepgram transcript
2024-03-07 20:31:44.103758 99.93% [DEBUG] dg_transcribe_glue.cpp:305 discarding empty deepgram transcript
2024-03-07 20:31:44.303757 99.93% [DEBUG] dg_transcribe_glue.cpp:305 discarding empty deepgram transcript
2024-03-07 20:31:45.283757 99.93% [DEBUG] dg_transcribe_glue.cpp:305 discarding empty deepgram transcript
2024-03-07 20:31:46.463765 99.90% [DEBUG] dg_transcribe_glue.cpp:309 deepgram message: {"type":"Results","channel_index":[0,1],"duration":2.0189373,"start":2.26,"is_final":false,"speech_final":false,"channel":{"alternatives":[{"transcript":"हां बोलो","confidence":0.9941406,"words":[{"word":"हां","start":2.26,"end":2.76,"confidence":0.84228516,"punctuated_word":"हां"},{"word":"बोलो","start":3.2694688,"end":3.7694688,"confidence":0.9941406,"punctuated_word":"बोलो"}]}]},"metadata":{"request_id":"b0a43754-4336-4821-81ec-2eeda6d5bb4d","model_info":{"name":"general-enhanced","version":"1982-11-21.24709","arch":"polaris"},"model_uuid":"83d5f3ce-9a16-4eb9-affe-83bd9ad10f69"}}
2024-03-07 20:31:47.443757 99.90% [DEBUG] dg_transcribe_glue.cpp:309 deepgram message: {"type":"Results","channel_index":[0,1],"duration":3.0189373,"start":2.26,"is_final":false,"speech_final":false,"channel":{"alternatives":[{"transcript":"हां बोलो क्या काम है?","confidence":0.89453125,"words":[{"word":"हां","start":3.7792401,"end":3.97914,"confidence":0.78222656,"punctuated_word":"हां"},{"word":"बोलो","start":3.97914,"end":4.33896,"confidence":0.9980469,"punctuated_word":"बोलो"},{"word":"क्या","start":4.33896,"end":4.5388603,"confidence":0.8520508,"punctuated_word":"क्या"},{"word":"काम","start":4.5388603,"end":4.81872,"confidence":0.9995117,"punctuated_word":"काम"},{"word":"है","start":4.81872,"end":5.2789373,"confidence":0.89453125,"punctuated_word":"है?"}]}]},"metadata":{"request_id":"b0a43754-4336-4821-81ec-2eeda6d5bb4d","model_info":{"name":"general-enhanced","version":"1982-11-21.24709","arch":"polaris"},"model_uuid":"83d5f3ce-9a16-4eb9-affe-83bd9ad10f69"}}
2024-03-07 20:31:47.623757 99.90% [DEBUG] dg_transcribe_glue.cpp:309 deepgram message: {"type":"Results","channel_index":[0,1],"duration":3.2500002,"start":2.26,"is_final":true,"speech_final":true,"channel":{"alternatives":[{"transcript":"हां बोलो क्या काम है?","confidence":0.9238281,"words":[{"word":"हां","start":3.7792401,"end":3.97914,"confidence":0.78222656,"punctuated_word":"हां"},{"word":"बोलो","start":3.97914,"end":4.33896,"confidence":0.99853516,"punctuated_word":"बोलो"},{"word":"क्या","start":4.33896,"end":4.5388603,"confidence":0.8540039,"punctuated_word":"क्या"},{"word":"काम","start":4.5388603,"end":4.81872,"confidence":0.9995117,"punctuated_word":"काम"},{"word":"है","start":4.81872,"end":5.31872,"confidence":0.9238281,"punctuated_word":"है?"}]}]},"metadata":{"request_id":"b0a43754-4336-4821-81ec-2eeda6d5bb4d","model_info":{"name":"general-enhanced","version":"1982-11-21.24709","arch":"polaris"},"model_uuid":"83d5f3ce-9a16-4eb9-affe-83bd9ad10f69"}}