search

ufal / whisper_streaming

Whisper realtime streaming for long speech-to-text transcription and translation
MIT License
2.09k stars 254 forks source link

Trying to reduce transcribing latency #135

Closed Sarankumar-Gembrill closed 3 weeks ago

Sarankumar-Gembrill commented 3 weeks ago

Hey,

I am trying to implementing whisper streaming code to transcribe the speech to text, and it is working.

But facing latency issues like below. While starting it is transcribing 1.01 seconds from 0.00 but for the next time it is transcribing 13.86 seconds from 0.00. I want the transcribe for every 1 or 2 seconds and I need the output also in the same way ( It came nearly after 30 seconds of speech: 0 4440 Hello, this is Saran Kumar. Good morning.)

INFO Whisper is warmed up. INFO Listening on('localhost', 5000) INFO Connected to client on ('127.0.0.1', 57184) DEBUG PROMPT: DEBUG CONTEXT: DEBUG transcribing 1.01 seconds from 0.00 DEBUG >>>>COMPLETE NOW: (None, None, '') DEBUG INCOMPLETE: (0.0, 0.92, ' hello') DEBUG len of buffer now: 1.01 DEBUG No text in this segment DEBUG PROMPT: DEBUG CONTEXT: DEBUG transcribing 13.86 seconds from 0.00 DEBUG >>>>COMPLETE NOW: (None, None, '') DEBUG INCOMPLETE: (0.0, 4.44, ' Hello, this is Saran Kumar. Good morning.') DEBUG len of buffer now: 13.86 DEBUG No text in this segment DEBUG PROMPT: DEBUG CONTEXT: DEBUG transcribing 27.85 seconds from 0.00 DEBUG >>>>COMPLETE NOW: (0.0, 4.44, ' Hello, this is Saran Kumar. Good morning.') DEBUG INCOMPLETE: (None, None, '') DEBUG --- not enough segments to chunk DEBUG chunking segment DEBUG len of buffer now: 27.85 0 4440 Hello, this is Saran Kumar. Good morning. DEBUG PROMPT: DEBUG CONTEXT: Hello, this is Saran Kumar. Good morning. DEBUG transcribing 41.89 seconds from 0.00 DEBUG >>>>COMPLETE NOW: (None, None, '') DEBUG INCOMPLETE: (39.74, 41.86, ' Good morning.') DEBUG --- segment chunked at 4.44 DEBUG chunking segment DEBUG len of buffer now: 37.45 DEBUG No text in this segment DEBUG PROMPT: Hello, this is Saran Kumar. Good DEBUG CONTEXT: morning. DEBUG transcribing 65.24 seconds from 4.44

Can you please help me to enhance this code to give me the results for every 1 or 2 seconds.

Gldkslfmsd commented 3 weeks ago

how big audio chunks do you send to the server? can you try simulating from file with whisper_online.py ? and with unaware mode? do you use gpu and faster whisper?

Sarankumar-Gembrill commented 3 weeks ago

I am using CPU and Faster whisper in backend. I am trying with min-chunk-size=1.0 seconds and using the same whisper_online.py file.

Gldkslfmsd commented 3 weeks ago

CPU is slow, it's known limitation. You need GPU