Open attsion opened 1 year ago
I'm also interested to know how use this with real-time audio streams
I managed to record audio chunks in real time using NAudio and iterate them as they were available. It seems to slow to handle them. Is there any way to speed things up?
https://github.com/AwesomeYuer/whisper.NAudio.NET is an example but it does seem too slow for real time with this repo.
https://github.com/Const-me/Whisper/tree/master/Examples/MicrophoneCS This other implementation is much faster but it doesn't use naudio loopback (doesn't detect silence... it seems to basically chuck audio depending on what you set the capture max and min duration to be)
Nice options are provided here, I am thinking that the best solution will be to have something like:
Having 2 configurable values: IntervalTime = The time needed for one processing (e.g. default to 5 seconds) OverlapTime= The time which will be processed twice in order to have continuity. (Default to 1 second)
During merge we need to keep in mind that exact the end of the segment might be gibberish, the same as the begging of the new segment (as it can be a word which is cut in half in any part).
This way, the context will be maintained as if you just cut randomly and process everything it can end up in the middle of the word and that cannot be recognized.
On the other hand, if we're always processing everything from 0 to CurrentTime, that will become too slow.
In order to improve quality, we can increase the overlap time.
Ideally, there would be this capability directly in this package, and anyone would be able to use NAudio or other stream provider to call some library (e.g. some PushStream similar to Azure Cognitive Service's PushStreams: https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/7e61fcb022f5dd75cfaf579703f8c92ad83317b0/samples/csharp/sharedcontent/console/speech_recognition_samples.cs#LL352C26-L352C26 )
+1
thank u