search

synesthesiam / homeassistant-satellite

Streaming audio satellite for Home Assistant
MIT License
188 stars 23 forks source link

Recording debug audio results in hundreds of empty wav files #59

Open vhsdream opened 1 year ago

vhsdream commented 1 year ago

For some reason this just started this morning. In Home Assistant the only setting I changed was the threshold (changed it back to 1 from 0.7). Restarted all add-ons (openwakeword, whisper and piper) and the satellite on the pi 3B. It stopped detecting the wakeword.

I attempted to debug by recording the input. Killed the process with Ctrl-C and checked the recording dir and I found hundreds of wav files, all created at the same time and having the same size (1004 bytes), but all were empty.

I am up-to-date with the latest git version.

Command:

script/run --host 10.10.40.50 --token-file token --pipeline nabu --mic-device plughw:CARD=S330 --snd-device \ 
plughw:CARD=S330 --awake-sound sounds/awake.wav --done-sound sounds/done.wav --vad webrtcvad \ 
--noise-suppression 4 --volume 0.25 --auto-gain 30 --debug --debug-recording-dir ./

Debug output showing it detect speech but fail to detect the wakeword 'ok nabu':

DEBUG:main:Waiting for speech Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono WARNING:root:Speech detected DEBUG:root:send_json() message={'type': 'assist_pipeline/pipeline/list', 'id': 1} DEBUG:root:send_and_subscribe_json() response={'id': 1, 'type': 'result', 'success': True, 'result': {'pipelines': [{'conversation_engine': 'homeassistant', 'conversation_language': 'en', 'language': 'en', 'name': 'Home Assistant', 'stt_engine': 'stt.faster_whisper', 'stt_language': 'en', 'tts_engine': 'tts.piper', 'tts_language': 'en_US', 'tts_voice': 'en_US-libritts-high', 'wake_word_entity': 'wake_word.openwakeword', 'wake_word_id': 'hey_rhasspy_v0.1', 'id': '01gzrpqhyz8kf7xeb3pbyq9d5z'}, {'conversation_engine': 'homeassistant', 'conversation_language': 'en', 'language': 'en', 'name': 'alexa', 'stt_engine': 'stt.faster_whisper', 'stt_language': 'en', 'tts_engine': 'tts.piper', 'tts_language': 'en_US', 'tts_voice': 'en_US-libritts_r-medium', 'wake_word_entity': 'wake_word.openwakeword', 'wake_word_id': 'alexa_v0.1', 'id': '01hcqwcgzse4fckgsy36q2h9r0'}, {'conversation_engine': 'homeassistant', 'conversation_language': 'en', 'language': 'en', 'name': 'nabu', 'stt_engine': 'stt.faster_whisper', 'stt_language': 'en', 'tts_engine': 'tts.piper', 'tts_language': 'en_US', 'tts_voice': 'en_US-hfc_male-medium', 'wake_word_entity': 'wake_word.openwakeword', 'wake_word_id': 'ok_nabu_v0.1', 'id': '01hd21mvvcxrax7pxh48v4q3hc'}, {'conversation_engine': '0877cbc7fb84a3b6f752637e6ee1fd47', 'conversation_language': 'en-CA', 'language': 'en', 'name': 'Google', 'stt_engine': 'stt.faster_whisper', 'stt_language': 'en', 'tts_engine': 'tts.piper', 'tts_language': 'en_US', 'tts_voice': 'en_US-amy-low', 'wake_word_entity': None, 'wake_word_id': None, 'id': '01hd7jj1rv725d7mb04rw02d04'}, {'conversation_engine': 'homeassistant', 'conversation_language': 'en', 'language': 'en', 'name': 'Homer', 'stt_engine': 'stt.faster_whisper', 'stt_language': 'en', 'tts_engine': 'tts.piper', 'tts_language': 'en_US', 'tts_voice': 'en_US-hfc_male-medium', 'wake_word_entity': 'wake_word.openwakeword', 'wake_word_id': 'Hey_Homer', 'id': '01hdj5wb3m1fmsn3e8mj80dcfg'}], 'preferred_pipeline': '01gzrpqhyz8kf7xeb3pbyq9d5z'}} DEBUG:homeassistant_satellite.remote:{'id': 1, 'type': 'result', 'success': True, 'result': {'pipelines': [{'conversation_engine': 'homeassistant', 'conversation_language': 'en', 'language': 'en', 'name': 'Home Assistant', 'stt_engine': 'stt.faster_whisper', 'stt_language': 'en', 'tts_engine': 'tts.piper', 'tts_language': 'en_US', 'tts_voice': 'en_US-libritts-high', 'wake_word_entity': 'wake_word.openwakeword', 'wake_word_id': 'hey_rhasspy_v0.1', 'id': '01gzrpqhyz8kf7xeb3pbyq9d5z'}, {'conversation_engine': 'homeassistant', 'conversation_language': 'en', 'language': 'en', 'name': 'alexa', 'stt_engine': 'stt.faster_whisper', 'stt_language': 'en', 'tts_engine': 'tts.piper', 'tts_language': 'en_US', 'tts_voice': 'en_US-libritts_r-medium', 'wake_word_entity': 'wake_word.openwakeword', 'wake_word_id': 'alexa_v0.1', 'id': '01hcqwcgzse4fckgsy36q2h9r0'}, {'conversation_engine': 'homeassistant', 'conversation_language': 'en', 'language': 'en', 'name': 'nabu', 'stt_engine': 'stt.faster_whisper', 'stt_language': 'en', 'tts_engine': 'tts.piper', 'tts_language': 'en_US', 'tts_voice': 'en_US-hfc_male-medium', 'wake_word_entity': 'wake_word.openwakeword', 'wake_word_id': 'ok_nabu_v0.1', 'id': '01hd21mvvcxrax7pxh48v4q3hc'}, {'conversation_engine': '0877cbc7fb84a3b6f752637e6ee1fd47', 'conversation_language': 'en-CA', 'language': 'en', 'name': 'Google', 'stt_engine': 'stt.faster_whisper', 'stt_language': 'en', 'tts_engine': 'tts.piper', 'tts_language': 'en_US', 'tts_voice': 'en_US-amy-low', 'wake_word_entity': None, 'wake_word_id': None, 'id': '01hd7jj1rv725d7mb04rw02d04'}, {'conversation_engine': 'homeassistant', 'conversation_language': 'en', 'language': 'en', 'name': 'Homer', 'stt_engine': 'stt.faster_whisper', 'stt_language': 'en', 'tts_engine': 'tts.piper', 'tts_language': 'en_US', 'tts_voice': 'en_US-hfc_male-medium', 'wake_word_entity': 'wake_word.openwakeword', 'wake_word_id': 'Hey_Homer', 'id': '01hdj5wb3m1fmsn3e8mj80dcfg'}], 'preferred_pipeline': '01gzrpqhyz8kf7xeb3pbyq9d5z'}} DEBUG:root:send_json() message={'type': 'assist_pipeline/run', 'start_stage': 'wake_word', 'end_stage': 'tts', 'input': {'sample_rate': 16000, 'timeout': 3, 'audio_seconds_to_buffer': 0}, 'pipeline': '01hd21mvvcxrax7pxh48v4q3hc', 'id': 2} DEBUG:root:send_and_subscribe_json() response={'id': 2, 'type': 'result', 'success': True, 'result': None} DEBUG:homeassistant_satellite.remote:{'id': 2, 'type': 'result', 'success': True, 'result': None} DEBUG:root:send_and_subscribe_json() response={'id': 2, 'type': 'event', 'event': {'type': 'run-start', 'data': {'pipeline': '01hd21mvvcxrax7pxh48v4q3hc', 'language': 'en', 'runner_data': {'stt_binary_handler_id': 1, 'timeout': 300}}, 'timestamp': '2023-11-03T13:06:14.433317+00:00'}} DEBUG:homeassistant_satellite.remote:{'id': 2, 'type': 'event', 'event': {'type': 'run-start', 'data': {'pipeline': '01hd21mvvcxrax7pxh48v4q3hc', 'language': 'en', 'runner_data': {'stt_binary_handler_id': 1, 'timeout': 300}}, 'timestamp': '2023-11-03T13:06:14.433317+00:00'}} DEBUG:root:send_and_subscribe_json() response={'id': 2, 'type': 'event', 'event': {'type': 'wake_word-start', 'data': {'entity_id': 'wake_word.openwakeword', 'metadata': {'format': 'wav', 'codec': 'pcm', 'bit_rate': 16, 'sample_rate': 16000, 'channel': 1}, 'timeout': 3}, 'timestamp': '2023-11-03T13:06:14.433643+00:00'}} DEBUG:homeassistant_satellite.remote:{'id': 2, 'type': 'event', 'event': {'type': 'wake_word-start', 'data': {'entity_id': 'wake_word.openwakeword', 'metadata': {'format': 'wav', 'codec': 'pcm', 'bit_rate': 16, 'sample_rate': 16000, 'channel': 1}, 'timeout': 3}, 'timestamp': '2023-11-03T13:06:14.433643+00:00'}} DEBUG:main:wake_word-start {'entity_id': 'wake_word.openwakeword', 'metadata': {'format': 'wav', 'codec': 'pcm', 'bit_rate': 16, 'sample_rate': 16000, 'channel': 1}, 'timeout': 3} DEBUG:root:send_and_subscribe_json() response={'id': 2, 'type': 'event', 'event': {'type': 'error', 'data': {'code': 'wake-word-timeout', 'message': 'Wake word was not detected'}, 'timestamp': '2023-11-03T13:06:18.034528+00:00'}} DEBUG:homeassistant_satellite.remote:{'id': 2, 'type': 'event', 'event': {'type': 'error', 'data': {'code': 'wake-word-timeout', 'message': 'Wake word was not detected'}, 'timestamp': '2023-11-03T13:06:18.034528+00:00'}} DEBUG:main:error {'code': 'wake-word-timeout', 'message': 'Wake word was not detected'} DEBUG:root:send_and_subscribe_json() response={'id': 2, 'type': 'event', 'event': {'type': 'run-end', 'data': None, 'timestamp': '2023-11-03T13:06:18.034573+00:00'}} DEBUG:homeassistant_satellite.remote:{'id': 2, 'type': 'event', 'event': {'type': 'run-end', 'data': None, 'timestamp': '2023-11-03T13:06:18.034573+00:00'}} DEBUG:main:run-end None DEBUG:homeassistant_satellite.remote:Pipeline finished DEBUG:main:Waiting for speech DEBUG:root:send_json() message={'type': 'fire_event', 'event_type': 'homeassistant_satellite_event', 'event_data': {'satellite_name': 'voice', 'pipeline_event': {'type': 'run-end', 'data': None}}, 'id': 3} DEBUG:root:send_and_subscribe_json() response={'id': 3, 'type': 'result', 'success': True, 'result': {'context': {'id': '01HEAMN8A6QGVWWDQ8YHKY1VK4', 'parent_id': None, 'user_id': '55977c99c63946fbbfc3f490730d8a95'}}}

Recording-dir spammed with identical empty wav files:

satellite

Risbo6 commented 1 year ago

I have the exact same issue. I'm on commit a8124311751c2e64a2b9a1fea9a308bcd9c085af.

vhsdream commented 1 year ago

Small update to this. Functionality returns when I set the Threshold value in openwakeword to a number other than 1. Setting it back to 0.8 worked for me.

@Risbo6 what is your current threshold setting for openwakeword?

Risbo6 commented 1 year ago

Small update to this. Functionality returns when I set the Threshold value in openwakeword to a number other than 1. Setting it back to 0.8 worked for me.

@Risbo6 what is your current threshold setting for openwakeword?

I'm using porcupine and it happens no matter what threshold I use

kriskbx commented 1 year ago

I experience more or less the same issue but only when using pulseaudio. Those 1 byte files actually contain a little bit of data. Using sox '*' combined.wav I was able to combine and listen to them in full. Changing the threshold didn't help, unfortunately.