search

KoljaB / RealtimeSTT

A robust, efficient, low-latency speech-to-text library with advanced voice activity detection, wake word activation and instant transcription.
MIT License
1.5k stars 139 forks source link

Whisper as wakeword detection #24

Closed olegchomp closed 1 month ago

olegchomp commented 6 months ago

Hi! Is there any easy way to switch wakeword detection from porcupine to whisper?

KoljaB commented 6 months ago

Not really, I also really wouldn't recommend that. Whisper would have to nonstop transcribe, which would result in high GPU load / energy consumption compare to porcupine. Also it can't compete with a wakeword engine in terms of accuracy and detection speed.

I think for the future switching to openWakeWord could be the right thing to do. Need to do some tests before.

olegchomp commented 6 months ago

Sadly. As i understand porcupine have only cloud solution for training new words and openwakeword support only EN language.

KoljaB commented 6 months ago

Oh, didn't know openwakeword only supports english. That's disappointing, we really need a multilingual, trainable wake word solution. I started with a Whisper wake word implementation first but esp accuracy was rather disappointing. I feel currently there is no easy solution to this problem out there.

sailorsale commented 6 months ago

Yes, we do need multilingualism

KoljaB commented 1 month ago

OpenWakeword done with v0.2.0 Multilingualism can be done with spelling out the sounds phonetically with underscores separating each part. See OpenWakeword google collab notebook.