search

synesthesiam / homeassistant-satellite

Streaming audio satellite for Home Assistant
MIT License
187 stars 23 forks source link

Wakeword not being detected #6

Open HarvsG opened 1 year ago

HarvsG commented 1 year ago

The wake words "OK nabu" and "hey jarvis" are never detected. image

image

Command: script/run --host 192.168.0.30 --token IUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiI0NDredactedG9kg22mvNO --awake-sound sounds/awake.wav --done-sound sounds/done.wav --vad webrtcvad --debug

Hardware, raspberrypi 3b with Razer blackshark microphone + earphones headset. speaker-test -t wav -c 2 plays audio succesfully arecord --format=S16_LE --duration=5 --rate=16000 --file-type=raw out.raw records good quality audio

Homeassistant running on raspberry pi 4.

Using --debug-recording-dir <DIR> successfully records audio that I can playback and listen to that is of good quality.

OpenWakeWord Debug Logs - lots of connections and disconnections (wired network)

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service openwakeword: starting
s6-rc: info: service openwakeword successfully started
s6-rc: info: service discovery: starting
DEBUG:root:Namespace(uri='tcp://0.0.0.0:10400', models_dir=PosixPath('/usr/local/lib/python3.11/dist-packages/wyoming_openwakeword/models'), custom_model_dir=['/share/openwakeword'], preload_model=['ok_nabu'], threshold=0.5, trigger_level=1, output_dir=None, debug=True, debug_probability=False, model=[])
DEBUG:root:Loading ok_nabu from /usr/local/lib/python3.11/dist-packages/wyoming_openwakeword/models/ok_nabu_v0.1.tflite
DEBUG:wyoming_openwakeword.handler:Started thread for ok_nabu
DEBUG:root:Loading /usr/local/lib/python3.11/dist-packages/wyoming_openwakeword/models/melspectrogram.tflite
DEBUG:root:Loading /usr/local/lib/python3.11/dist-packages/wyoming_openwakeword/models/embedding_model.tflite
INFO:root:Ready
DEBUG:wyoming_openwakeword.handler:Client connected: 2372413966959250
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372413966959250
[23:33:50] INFO: Successfully sent discovery information to Home Assistant.
s6-rc: info: service discovery successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
DEBUG:wyoming_openwakeword.handler:Client connected: 2372414857528512
DEBUG:root:Loading ok_nabu_v0.1 from /usr/local/lib/python3.11/dist-packages/wyoming_openwakeword/models/ok_nabu_v0.1.tflite
DEBUG:wyoming_openwakeword.handler:Started thread for ok_nabu_v0.1
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372414857528512
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372414857528512
DEBUG:wyoming_openwakeword.handler:Client connected: 2372418018292914
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372418018292914
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372418018292914
DEBUG:wyoming_openwakeword.handler:Client connected: 2372421131344396
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372421131344396
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372421131344396
DEBUG:wyoming_openwakeword.handler:Client connected: 2372424245476678
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372424245476678
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372424245476678
DEBUG:wyoming_openwakeword.handler:Client connected: 2372427366806785
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372427366806785
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372427366806785
DEBUG:wyoming_openwakeword.handler:Client connected: 2372430477170133
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372430477170133
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372430477170133
DEBUG:wyoming_openwakeword.handler:Client connected: 2372433635466737
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372433635466737
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372433635466737
DEBUG:wyoming_openwakeword.handler:Client connected: 2372436748400230
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372436748400230
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372436748400230
DEBUG:wyoming_openwakeword.handler:Client connected: 2372439864739777
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372439864739777
DEBUG:wyoming_openwakeword.handler:Client connected: 2372442225860357
DEBUG:wyoming_openwakeword.handler:Sent info to client: 2372442225860357
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372439864739777
DEBUG:wyoming_openwakeword.handler:Client connected: 2372442978494943
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372442978494943
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372442225860357
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372442978494943
DEBUG:wyoming_openwakeword.handler:Client connected: 2372446138718724
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372446138718724
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372446138718724
DEBUG:wyoming_openwakeword.handler:Client connected: 2372449253325951
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372449253325951
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372449253325951
DEBUG:wyoming_openwakeword.handler:Client connected: 2372455369511985
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372455369511985
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372455369511985
DEBUG:wyoming_openwakeword.handler:Client connected: 2372458505329691
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372458505329691
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372458505329691
DEBUG:wyoming_openwakeword.handler:Client connected: 2372461624311120
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372461624311120
DEBUG:wyoming_openwakeword.handler:Client connected: 2372471783886166
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372471783886166
DEBUG:wyoming_openwakeword.handler:Client connected: 2372474472219707
DEBUG:wyoming_openwakeword.handler:Sent info to client: 2372474472219707
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372474472219707
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372471783886166
DEBUG:wyoming_openwakeword.handler:Client connected: 2372489405350065
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372489405350065
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372489405350065
DEBUG:wyoming_openwakeword.handler:Client connected: 2372508261553845
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372508261553845
DEBUG:wyoming_openwakeword.handler:Client connected: 2372510596387034
DEBUG:wyoming_openwakeword.handler:Sent info to client: 2372510596387034
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372510596387034
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372508261553845
DEBUG:wyoming_openwakeword.handler:Client connected: 2372516750043089
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372516750043089
DEBUG:wyoming_openwakeword.handler:Client disconnected: 2372516750043089
DEBUG:wyoming_openwakeword.handler:Client connected: 2372520505958312
DEBUG:wyoming_openwakeword.handler:Receiving audio from client: 2372520505958312
HarvsG commented 1 year ago

Ok so it seems like just a very low sensitivity, getting like 1 in 20. Likely a openwakeword problem, I'll do some more testing tomorrow.

HarvsG commented 1 year ago

script/run --host 192.168.0.30 --token eyJhbGciOiJIUzI1redacted --done-sound sounds/done.wav --vad webrtcvad --debug --mic-device plughw:CARD=Device --debug-recording-dir records --noise-suppression 2 --auto-gain 25 --volume-multiplier 4

Ok, so I'm getting about a 50% hit rate with this command, using a tiny usb omnidirectional microphone and thrshold set in OpenWakeWord to 0.01

I have never got a second wake word to work.

HarvsG commented 1 year ago

I had to remove the awake sound as I was hitting this issue: https://github.com/synesthesiam/homeassistant-satellite/pull/1

HarvsG commented 1 year ago

script/run --host 192.168.0.30 --token eyJhredactedr94zQ --done-sound sounds/done.wav --vad webrtcvad --debug --mic-device plughw:CARD=Device --debug-recording-dir records --noise-suppression 2 --auto-gain 25 --volume-multiplier 6

So this command is working well with this microphone: https://www.amazon.co.uk/Recording-Microphone-Omnidirectional-Portable-Conferencing-BLACK/dp/B088D6SJBQ/ref=sr_1_3?crid=2BBB3RVD5RGEK&keywords=usb+microphone+raspberry+pi&qid=1697203109&sprefix=usb+microphone+rasp%2Caps%2C73&sr=8-3

Getting about a 70% pickup rate with a threshold of 0.5.

donburch888 commented 1 year ago

For anyone else with the same issue...

I also had problems with wakeword being detected using a cheap USB microphone. I used the "--debug-recording-dir

" option (mentioned at the very bottom of the homeassistant-satellite page) to listen to the recording and found quite bad audio effects which the audio enhancement options weren't able to clean up :-(

I swapped to a different microphone, and audio is now clear and no enhancements necessary.