rhasspy / wyoming-satellite

Remote voice satellite using Wyoming protocol
MIT License
616 stars 91 forks source link

satellite gets stuck after only 1 wake word detection #154

Open jerome83136 opened 5 months ago

jerome83136 commented 5 months ago

Hello,

I built wyoming-satellite and I run it this way:

/opt/wyoming-satellite# script/run --name "my satellite" --uri tcp://0.0.0.0:10700 --mic-command "arecord -D plughw:CARD=seeed8micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw" --snd-command "aplay -D plughw:CARD=seeed8micvoicec,DEV=1 -r 22050 -c 1 -f S16_LE -t raw" --awake-wav sounds/awake.wav --detection-command examples/commands/detection.sh --wake-refractory-seconds 0 --debug

I'm using a custom wakeword (Google colab) and it works fine; but it only works "once" I mean:

  1. I start the satellite process with the above command line
  2. I wait for it to start
  3. Then I say the wakeword "hé chou pee"
  4. The satellite activates, plays the sound (-awake-wav) and wait for my command
  5. The command is recognized (STT --> ASR --> Intent) and gets executed
  6. The audio feedback (TTS) is sent back to the satellite and I can hear it

And that's it. I am unable to get the wakeword activating again without killing and restarting the satellite process. After restarting; it works (for 1 time) etc...

NB: I had the same result when using standard wakewords (hey_rhasspy, hey_jarvis, etc...)

Here are DEBUG logs of my satellite:

DEBUG:root:Namespace(mic_uri=None, mic_command='arecord -D plughw:CARD=seeed8micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw', mic_command_rate=16000, mic_command_width=2, mic_command_channels=1, mic_command_samples_per_chunk=1024, mic_volume_multiplier=1.0, mic_noise_suppression=0, mic_auto_gain=0, mic_seconds_to_mute_after_awake_wav=0.5, mic_no_mute_during_awake_wav=False, mic_channel_index=None, snd_uri=None, snd_command='aplay -D plughw:CARD=seeed8micvoicec,DEV=1 -r 22050 -c 1 -f S16_LE -t raw', snd_command_rate=22050, snd_command_width=2, snd_command_channels=1, snd_volume_multiplier=1.0, wake_uri=None, wake_word_name=[], wake_command=None, wake_command_rate=16000, wake_command_width=2, wake_command_channels=1, wake_refractory_seconds=0.0, vad=False, vad_threshold=0.5, vad_trigger_level=1, vad_buffer_seconds=2, vad_wake_word_timeout=5.0, event_uri=None, startup_command=None, detect_command=None, detection_command='examples/commands/detection.sh', transcript_command=None, stt_start_command=None, stt_stop_command=None, synthesize_command=None, tts_start_command=None, tts_stop_command=None, tts_played_command=None, streaming_start_command=None, streaming_stop_command=None, error_command=None, connected_command=None, disconnected_command=None, awake_wav='sounds/awake.wav', done_wav=None, uri='tcp://0.0.0.0:10700', name='my satellite', area=None, no_zeroconf=False, zeroconf_name=None, zeroconf_host=None, debug_recording_dir=None, debug=True, log_format='%(levelname)s:%(name)s:%(message)s')
INFO:root:Ready
DEBUG:root:Detected IP: 192.168.1.244
DEBUG:root:Zeroconf discovery enabled (name=6c21a2437eae, host=None)
DEBUG:root:Connecting to mic service: ['arecord', '-D', 'plughw:CARD=seeed8micvoicec,DEV=0', '-r', '16000', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
DEBUG:root:Connecting to snd service: ['aplay', '-D', 'plughw:CARD=seeed8micvoicec,DEV=1', '-r', '22050', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
INFO:root:Connected to services
DEBUG:root:Connected to mic service
Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
DEBUG:root:Server set: 34436877217185
INFO:root:Connected to server
INFO:root:Streaming audio
DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'wake', 'end_stage': 'tts', 'restart_on_end': True, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
DEBUG:root:Ping enabled

DEBUG:root:Wake word detected
DEBUG:root:Running ['examples/commands/detection.sh']
Wake word detected: éh_chou_pee
DEBUG:root:Muting microphone for 0.8995918367346939 second(s)
DEBUG:root:Connected to snd service
Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
DEBUG:root:Unmuted microphone
DEBUG:root:Event(type='transcript', data={'text': ' Allume la cuisine !'}, payload=None)
INFO:root:Streaming audio
DEBUG:root:Event(type='synthesize', data={'text': 'Allumé', 'voice': {'name': 'fr_FR-siwis-low'}}, payload=None)
DEBUG:root:Connected to snd service
Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono

What did I do wrong please ?

Hardware:

Thank you for your help :)

Best regards

jerome83136 commented 5 months ago

Hi all,

After compiling required libs for activating VAD; I noticed that my voice is well detected

Here is the command I'm using with VAD argument: /opt/wyoming-satellite# script/run --name "my satellite" --uri tcp://0.0.0.0:10700 --mic-command "arecord -D plughw:CARD=seeed8micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw" --snd-command "aplay -D plughw:CARD=seeed8micvoicec,DEV=1 -r 22050 -c 1 -f S16_LE -t raw" --awake-wav sounds/awake.wav --detection-command examples/commands/detection.sh --wake-refractory-seconds 0 --vad --debug

And here are the logs whowing a successful wakeword recognition:

DEBUG:root:Ping enabled
INFO:root:Waiting for speech
INFO:root:Streaming audio
DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'wake', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
DEBUG:root:Wake word detected
DEBUG:root:Running ['examples/commands/detection.sh']
Wake word detected: éh_chou_pee
DEBUG:root:Muting microphone for 0.8995918367346939 second(s)
DEBUG:root:Connected to snd service
Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
DEBUG:root:Unmuted microphone
DEBUG:root:Event(type='transcript', data={'text': ' Éteins la cuisine.'}, payload=None)
DEBUG:root:Event(type='synthesize', data={'text': 'Éteint', 'voice': {'name': 'fr_FR-siwis-low'}}, payload=None)
DEBUG:root:Connected to snd service
Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
INFO:root:Streaming audio
DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'wake', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
INFO:root:Waiting for speech

And here; same logs showing my attempts to trigger wakeword again with no luck: (we can see the audio stream is detected when I speak; so it seems it is detected by the satellite, but the wakeword never gets triggered)

INFO:root:Waiting for speech
INFO:root:Streaming audio
DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'wake', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
INFO:root:Waiting for speech
INFO:root:Streaming audio
DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'wake', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
INFO:root:Waiting for speech
INFO:root:Streaming audio
DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'wake', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)

If I want to detect a wakeword again; I still need to kill and restart the satellite process. Then it works only once and I have to restart the process again.

Do anyone has some idea to share on this so that I can get it fixed maybe ? :)

NB: I'm also wondering what is "restart_on_end" that shows up in the logs ? Is it supposed to be "True" to somehow "allow" the wakeword going in "detecting" state again ?

Thank you for your feedback :) Best regards

d1egoaz commented 5 months ago

I have the same issue, I installed everything yesterday.

Apr 17 14:28:21 pizero sudo[3807]: pam_unix(sudo:session): session opened for user root(uid=0) by diego(uid=1000)
Apr 17 14:28:24 pizero sudo[3807]: pam_unix(sudo:session): session closed for user root
Apr 17 14:28:29 pizero run[3794]: DEBUG:root:Detection(name='hey_jarvis_v0.1', timestamp=20400732344365)
Apr 17 14:28:29 pizero run[3794]: DEBUG:root:Streaming audio
Apr 17 14:28:29 pizero run[3794]: DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
Apr 17 14:28:29 pizero run[3794]: DEBUG:root:Running ['examples/commands/detection.sh']
Apr 17 14:28:29 pizero run[3810]: Wake word detected: hey_jarvis_v0.1
Apr 17 14:28:29 pizero run[3794]: DEBUG:root:Muting microphone for 0.8995918367346939 second(s)
Apr 17 14:28:29 pizero run[3794]: DEBUG:root:Connected to snd service
Apr 17 14:28:29 pizero run[3813]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Apr 17 14:28:30 pizero run[3794]: DEBUG:root:Unmuted microphone
d1egoaz commented 5 months ago

I've tried previous versions, but the result was the same.

I've tried:

Apr 17 15:02:33 pizero run[5230]: Wake word detected: hey_jarvis_v0.1
Apr 17 15:02:33 pizero run[5206]: DEBUG:root:Muting microphone for 0.8995918367346939 second(s)
Apr 17 15:02:33 pizero run[5206]: DEBUG:root:Connected to snd service
Apr 17 15:02:33 pizero run[5233]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Apr 17 15:02:34 pizero run[5206]: DEBUG:root:Unmuted microphone
jerome83136 commented 5 months ago

Hi all, I did some more tests with ESP32-S3-BOX-3 (tutorial link)

So the device is using the same Home Assistant setup than the one I used with wyoming-satellite: wyoming openwakeword (still with my custom wakeword)

Only difference is that I replaced wyoming-satellite by an SP32-S3-BOX-3

And with this setup; I'm able to trigger the wakeword several times without having to restart anything. So I think Home Assistant setup + wyoming-openwakeword are working correctly and I think there is a problem with the configuration of my wyoming-satellite. But what can be the problem ? Really need some feedback on this please :)

NB: In case it can help: I'm using docker version of Home Assistant and wyoming-openwakeword

Thank you for your help Best regards

d1egoaz commented 5 months ago

Additional context:

I'm following https://www.youtube.com/watch?v=eTKgc0YDCwE&list=WL&index=54 with the same hardware:

- Raspberry Pi Zero W 2
- SeeedStudio: 2-Mic Pi Hat 

Additional logs:

I added >>> to the examples/commands to make then easier to be viewed:

Apr 18 12:41:47 pizero systemd[1]: Started Wyoming Satellite.
Apr 18 12:41:47 pizero python3[4091]: INFO:root:Ready
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Namespace(mic_uri=None, mic_command='arecord -D plughw:CARD=seeed2micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw', mic_command_rate=16000, mic_command_width=2, mic_command_channels=1, mic_command_samples_per_chunk=1024, mic_volume_multiplier=1.0, mic_noise_suppression=2, mic_auto_gain=0, mic_seconds_to_mute_after_awake_wav=0.5, mic_no_mute_during_awake_wav=False, mic_channel_index=None, snd_uri=None, snd_command='aplay -D plughw:CARD=seeed2micvoicec,DEV=0 -r 22050 -c 1 -f S16_LE -t raw', snd_command_rate=22050, snd_command_width=2, snd_command_channels=1, snd_volume_multiplier=1.0, wake_uri='tcp://127.0.0.1:10400', wake_word_name=[['hey_jarvis']], wake_command=None, wake_command_rate=16000, wake_command_width=2, wake_command_channels=1, wake_refractory_seconds=5.0, vad=False, vad_threshold=0.5, vad_trigger_level=1, vad_buffer_seconds=2, vad_wake_word_timeout=5.0, event_uri='tcp://127.0.0.1:10500', startup_command='examples/commands/startup.sh', detect_command=None, detection_command='examples/commands/detection.sh', transcript_command=None, stt_start_command='examples/commands/stt_start.sh', stt_stop_command='examples/commands/stt_stop.sh', synthesize_command='examples/commands/synthesize.sh', tts_start_command='examples/commands/tts_start.sh', tts_stop_command='examples/commands/tts_stop.sh', tts_played_command=None, streaming_start_command='examples/commands/streaming_start.sh', streaming_stop_command='examples/commands/streaming_stop.sh', error_command=None, connected_command=None, disconnected_command=None, awake_wav='sounds/awake.wav', done_wav='sounds/done.wav', uri='tcp://0.0.0.0:10700', name='Alfred Satellite', area=None, no_zeroconf=False, zeroconf_name=None, zeroconf_host=None, debug_recording_dir=None, debug=True, log_format='%(levelname)s:%(name)s:%(message)s')
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Running ['examples/commands/startup.sh']
Apr 18 12:41:48 pizero run[4099]: >>> Satellite has started.
Apr 18 12:41:48 pizero run[4095]: INFO:root:Ready
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Detected IP: 192.168.42.228
Apr 18 12:41:48 pizero run[4094]: DEBUG:root:Namespace(uri='tcp://127.0.0.1:10400', models_dir=PosixPath('/home/diego/wyoming-openwakeword/wyoming_openwakeword/models'), custom_model_dir=[], preload_model=['hey_jarvis'], threshold=0.5, trigger_level=1, output_dir=None, debug=True, log_format='%(levelname)s:%(name)s:%(message)s', debug_probability=False, version=False, model=[])
Apr 18 12:41:48 pizero run[4094]: DEBUG:root:Loading hey_jarvis_v0.1 from /home/diego/wyoming-openwakeword/wyoming_openwakeword/models/hey_jarvis_v0.1.tflite
Apr 18 12:41:48 pizero run[4094]: DEBUG:wyoming_openwakeword.handler:Started thread for hey_jarvis_v0.1
Apr 18 12:41:48 pizero run[4094]: DEBUG:root:Loading /home/diego/wyoming-openwakeword/wyoming_openwakeword/models/melspectrogram.tflite
Apr 18 12:41:48 pizero run[4094]: DEBUG:root:Loading /home/diego/wyoming-openwakeword/wyoming_openwakeword/models/embedding_model.tflite
Apr 18 12:41:48 pizero run[4094]: INFO:root:Ready
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Zeroconf discovery enabled (name=2ccf6704eefa, host=None)
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Connecting to mic service: ['arecord', '-D', 'plughw:CARD=seeed2micvoicec,DEV=0', '-r', '16000', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Connecting to snd service: ['aplay', '-D', 'plughw:CARD=seeed2micvoicec,DEV=0', '-r', '22050', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Connecting to wake service: tcp://127.0.0.1:10400
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Connecting to event service: tcp://127.0.0.1:10500
Apr 18 12:41:48 pizero run[4095]: INFO:root:Connected to services
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Using webrtc audio enhancements
Apr 18 12:41:48 pizero run[4094]: DEBUG:wyoming_openwakeword.handler:Client connected: 14883546340095
Apr 18 12:41:48 pizero run[4105]: Recording raw data 'stdin' : Signed 16 bit Little Endian,
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Connected to mic service
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Connected to wake service
Apr 18 12:41:48 pizero run[4105]: Rate 16000 Hz, Mono
Apr 18 12:41:48 pizero run[4094]: DEBUG:root:Loading hey_jarvis_v0.1 from /home/diego/wyoming-openwakeword/wyoming_openwakeword/models/hey_jarvis_v0.1.tflite
Apr 18 12:41:48 pizero run[4094]: DEBUG:wyoming_openwakeword.handler:Started thread for hey_jarvis_v0.1
Apr 18 12:41:49 pizero run[4094]: INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Apr 18 12:41:49 pizero run[4095]: DEBUG:root:Server set: 14884328193371
Apr 18 12:41:49 pizero run[4095]: INFO:root:Connected to server
Apr 18 12:41:49 pizero run[4095]: DEBUG:root:Running ['examples/commands/streaming_stop.sh']
Apr 18 12:41:49 pizero run[4108]: >>> Audio streaming to server has stopped
Apr 18 12:41:49 pizero run[4095]: INFO:root:Waiting for wake word
Apr 18 12:41:49 pizero run[4095]: DEBUG:root:Connected to event service
Apr 18 12:41:49 pizero run[4094]: DEBUG:root:Loading hey_jarvis_v0.1 from /home/diego/wyoming-openwakeword/wyoming_openwakeword/models/hey_jarvis_v0.1.tflite
Apr 18 12:41:49 pizero run[4094]: DEBUG:wyoming_openwakeword.handler:Started thread for hey_jarvis_v0.1
Apr 18 12:41:49 pizero run[4094]: DEBUG:wyoming_openwakeword.handler:Sent info to client: 14883546340095
Apr 18 12:41:51 pizero run[4095]: DEBUG:root:Ping enabled
Apr 18 12:42:06 pizero run[4094]: DEBUG:root:Triggered hey_jarvis_v0.1 (client=14883546340095)
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Detection(name='hey_jarvis_v0.1', timestamp=14900721294606)
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Streaming audio
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Running ['examples/commands/detection.sh']
Apr 18 12:42:06 pizero run[4094]: DEBUG:root:Triggered hey_jarvis_v0.1 (client=14883546340095)
Apr 18 12:42:06 pizero run[4111]: >>> Wake word detected: hey_jarvis_v0.1
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Muting microphone for 0.8995918367346939 second(s)
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Running ['examples/commands/streaming_start.sh']
Apr 18 12:42:06 pizero run[4094]: DEBUG:root:Triggered hey_jarvis_v0.1 (client=14883546340095)
Apr 18 12:42:06 pizero run[4114]: >>> Audio streaming to server has started
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Connected to snd service
Apr 18 12:42:06 pizero run[4116]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Apr 18 12:42:07 pizero run[4095]: DEBUG:root:Unmuted microphone

Only the called commands logs:

Apr 18 12:41:48 pizero run[4099]: >>> Satellite has started.
Apr 18 12:41:49 pizero run[4108]: >>> Audio streaming to server has stopped
Apr 18 12:42:06 pizero run[4111]: >>> Wake word detected: hey_jarvis_v0.1
Apr 18 12:42:06 pizero run[4114]: >>> Audio streaming to server has started

And this is my service definition:

[Unit]
Description=Wyoming Satellite
Wants=network-online.target
After=network-online.target
Requires=wyoming-openwakeword.service
Requires=2mic_leds.service

[Service]
Type=simple
ExecStart=/home/diego/wyoming-satellite/script/run \
        --name 'Alfred Satellite' \
        --uri 'tcp://0.0.0.0:10700' \
        --mic-command 'arecord -D plughw:CARD=seeed2micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw' \
        --snd-command 'aplay -D plughw:CARD=seeed2micvoicec,DEV=0 -r 22050 -c 1 -f S16_LE -t raw' \
        --mic-auto-gain 0 \
        --mic-noise-suppression 2 \
        --mic-volume-multiplier 1.0 \
        --wake-uri 'tcp://127.0.0.1:10400' \
        --wake-word-name 'hey_jarvis' \
        --event-uri 'tcp://127.0.0.1:10500' \
        --awake-wav sounds/awake.wav \
        --done-wav sounds/done.wav \
        --tts-stop-command 'true' \
        --detection-command examples/commands/detection.sh \
        --startup-command examples/commands/startup.sh \
        --streaming-start-command examples/commands/streaming_start.sh \
        --streaming-stop-command examples/commands/streaming_stop.sh \
        --synthesize-command examples/commands/synthesize.sh \
        --stt-start-command examples/commands/stt_start.sh \
        --stt-stop-command examples/commands/stt_stop.sh \
        --tts-start-command examples/commands/tts_start.sh \
        --tts-stop-command examples/commands/tts_stop.sh \
        --debug
WorkingDirectory=/home/diego/wyoming-satellite
Restart=always

Hopefully @synesthesiam can take a look at this 🙏

radunemerenco commented 5 months ago

I have the same setup, and I get into the same issue.

  1. I say the wake word
  2. I get the audio feedback
  3. I see it in the logs: Wake word detected: hey_jarvis_v0.1
  4. The last log is DEBUG:root:Unmuted microphone

The LED service doesn't even work. I was never able to turn the LED on

jerome83136 commented 5 months ago

Hi everyone Anyone able to help here maybe? 🙏 Thank you Best regards

richardoswald commented 4 months ago

Similar issue here, in debug mode it only responds to wake word one time and gets stuck. The satellite is configured in Home Assistant. I'm using most of the defaults from the tutorial since I have the same hardware wyoming-satellite v1.2.0 wyoming-openwakeword v1.10.0 Home Assistant v2024.5.4 (docker)

richard@wysat:~ $ journalctl -u wyoming-openwakeword.service -f
-- Journal begins at Mon 2024-03-11 20:13:58 CDT. --
May 18 00:47:51 wysat run[469]: DEBUG:root:Loading /home/richard/wyoming-openwakeword/wyoming_openwakeword/models/embedding_model.tflite
May 18 00:47:51 wysat run[469]: INFO:root:Ready
May 18 00:47:56 wysat run[469]: DEBUG:wyoming_openwakeword.handler:Client connected: 20644978897
May 18 00:47:56 wysat run[469]: DEBUG:root:Loading hey_jarvis_v0.1 from /home/richard/wyoming-openwakeword/wyoming_openwakeword/models/hey_jarvis_v0.1.tflite
May 18 00:47:56 wysat run[469]: DEBUG:wyoming_openwakeword.handler:Started thread for hey_jarvis_v0.1
May 18 00:47:57 wysat run[469]: INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
May 18 00:47:58 wysat run[469]: DEBUG:wyoming_openwakeword.handler:Sent info to client: 20644978897
May 18 00:48:03 wysat run[469]: DEBUG:root:Loading hey_jarvis_v0.1 from /home/richard/wyoming-openwakeword/wyoming_openwakeword/models/hey_jarvis_v0.1.tflite
May 18 00:48:03 wysat run[469]: DEBUG:wyoming_openwakeword.handler:Started thread for hey_jarvis_v0.1
May 18 00:48:03 wysat run[469]: DEBUG:wyoming_openwakeword.handler:Sent info to client: 20644978897
May 18 00:49:31 wysat run[469]: DEBUG:root:Triggered hey_jarvis_v0.1 (client=20644978897)
May 18 00:49:31 wysat run[469]: DEBUG:root:Triggered hey_jarvis_v0.1 (client=20644978897)
May 18 00:49:31 wysat run[469]: DEBUG:root:Triggered hey_jarvis_v0.1 (client=20644978897)
^C
richard@wysat:~ $ journalctl -u wyoming-satellite.service -f
-- Journal begins at Mon 2024-03-11 20:13:58 CDT. --
May 18 00:47:56 wysat run[644]: Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
May 18 00:47:56 wysat run[642]: DEBUG:root:Connected to mic service
May 18 00:47:56 wysat run[642]: DEBUG:root:Connected to wake service
May 18 00:48:03 wysat run[642]: DEBUG:root:Server set: 27291338790
May 18 00:48:03 wysat run[642]: INFO:root:Connected to server
May 18 00:48:03 wysat run[642]: INFO:root:Waiting for wake word
May 18 00:48:05 wysat run[642]: DEBUG:root:Ping enabled
May 18 00:49:31 wysat run[642]: DEBUG:root:Detection(name='hey_jarvis_v0.1', timestamp=102095540476)
May 18 00:49:31 wysat run[642]: DEBUG:root:Streaming audio
May 18 00:49:31 wysat run[642]: DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
betodaviola commented 3 months ago

Same problem here, trying to fix for a couple of days now with no success. Similarly to some people on this thread, I am using this page from the docs since that is the equipment I have (raspberry pi 0 2w and respeaker 2-mic pihat). In the logs bellow you can see the wake word triggering the first time I use it, but then it stops working:

$ sudo journalctl -u wyoming-satellite.service -f:
-- Journal begins at Thu 2024-07-04 01:14:43 BST. --
Jul 09 05:12:06 spoodnik1 run[1139]: INFO:root:Connected to services
Jul 09 05:12:06 spoodnik1 run[1139]: DEBUG:root:Using webrtc audio enhancements
Jul 09 05:12:06 spoodnik1 run[1141]: Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
Jul 09 05:12:06 spoodnik1 run[1139]: DEBUG:root:Connected to mic service
Jul 09 05:12:06 spoodnik1 run[1139]: DEBUG:root:Connected to wake service
Jul 09 05:12:07 spoodnik1 run[1139]: DEBUG:root:Server set: 1431054424930
Jul 09 05:12:07 spoodnik1 run[1139]: INFO:root:Connected to server
Jul 09 05:12:07 spoodnik1 run[1139]: INFO:root:Waiting for wake word
Jul 09 05:12:07 spoodnik1 run[1139]: DEBUG:root:Connection cancelled: 1431193952790
Jul 09 05:12:09 spoodnik1 run[1139]: DEBUG:root:Ping enabled
Jul 09 05:12:15 spoodnik1 run[1139]: DEBUG:root:Connection cancelled: 1439199064354
Jul 09 05:12:23 spoodnik1 run[1139]: DEBUG:root:Connection cancelled: 1447203942010
Jul 09 05:12:31 spoodnik1 run[1139]: DEBUG:root:Connection cancelled: 1455209925458
Jul 09 05:12:35 spoodnik1 run[1139]: DEBUG:root:Detection(name='ok_nabu_v0.1', timestamp=1458797918622, speaker=None)
Jul 09 05:12:35 spoodnik1 run[1139]: DEBUG:root:Streaming audio
Jul 09 05:12:35 spoodnik1 run[1139]: DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 16000, 'width': 2, 'channels': 1}}, payload=None)
Jul 09 05:12:39 spoodnik1 run[1139]: DEBUG:root:Connection cancelled: 1463215630900
Jul 09 05:12:47 spoodnik1 run[1139]: DEBUG:root:Connection cancelled: 1471219517763
Jul 09 05:12:55 spoodnik1 run[1139]: DEBUG:root:Connection cancelled: 1479229214886
Jul 09 05:13:03 spoodnik1 run[1139]: DEBUG:root:Connection cancelled: 1487227850808
Jul 09 05:13:11 spoodnik1 run[1139]: DEBUG:root:Connection cancelled: 1495233622580
Jul 09 05:13:19 spoodnik1 run[1139]: DEBUG:root:Connection cancelled: 1503238541588
Jul 09 05:13:27 spoodnik1 run[1139]: DEBUG:root:Connection cancelled: 1511243289506

My wyoming-satellite.service file:

[Unit]
Description=Wyoming Satellite
Wants=network-online.target
After=network-online.target
Requires=wyoming-openwakeword.service

[Service]
Type=simple
ExecStart=/home/spoodnik1/wyoming-satellite/script/run \
        --debug \
        --name 'spoodnik1' \
        --uri 'tcp://0.0.0.0:10700' \
        --mic-command 'arecord -D plughw:CARD=seeed2micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw' \
        --snd-command 'aplay -D plughw:CARD=seeed2micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw' \
        --snd-command-rate 16000 \
        --snd-volume-multiplier 0.5 \
        --mic-auto-gain 5 \
        --mic-noise-suppression 2 \
        --wake-uri 'tcp://127.0.0.1:10400' \
        --wake-refractory-seconds 5 \
        --wake-word-name 'ok_nabu'
WorkingDirectory=/home/spoodnik1/wyoming-satellite
Restart=always
RestartSec=1

[Install]
WantedBy=default.target
jerome83136 commented 3 months ago

Hi @synesthesiam, It seems there are several people facing the "keyword spotting only work once" issue

Can we kindly ask you for some support on this please? 🙏 Best regards

TomasKliner commented 3 months ago

Hello, I had a similliar issue, but just getting detected once, without ever executing any commands. I managed to locate the problem with Speech To Text in homeassistant logs.

Steps I took to fix it:

  1. installing addons Piper, Whisper and openWakeWord
  2. Adding them in integrations.
  3. Changing Speech To Text option to faster-whisper (inside settings -> voice assistant)

Not sure if all of the above is needed, but the core of the issue for me was not having a Speech to text agent configured, which made the service stuck in listening mode.

87 I suppose this could be the same issue as I had.

betodaviola commented 3 months ago

Hello, I had a similliar issue, but just getting detected once, without ever executing any commands. I managed to locate the problem with Speech To Text in homeassistant logs.

Steps I took to fix it:

1. installing addons **Piper**, **Whisper** and **openWakeWord**

2. Adding them in **integrations**.

3. Changing Speech To Text option to **faster-whisper** (inside settings -> voice assistant)

Not sure if all of the above is needed, but the core of the issue for me was not having a Speech to text agent configured, which made the service stuck in listening mode.

87 I suppose this could be the same issue as I had.

Thank you for sharing this. I will research about it and possibly try your solution today or tomorrow and report back. The logs you refer too are just the one in settings >> system >> logs? I don't seem to have any error regarding to speech-to-text in mine. DO you have any idea why these integrations need to be installed on the HA side of things? I thought that the satellite should take care of these things on their end, which is why I was trying to get it ready to go before working on the HA side of things. I am trying to figure out if we had indeed the same error

TomasKliner commented 3 months ago

First, I enabled logging on the Wyoming integration and then recreated the bug. Afterward, I went through the 'home-assistant.log' file, but it seems to be the same thing as you suggested. (Im fairly new to HA)

To be honest, I'm not entirely sure how it is intended to work in Home Assistant. My guess is you only need to have any speech-to-text service set up in the voice assistant settings.

ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved (None)
Traceback (most recent call last):

 File "/usr/src/homeassistant/homeassistant/components/assist_pipeline/__init__.py", line 123, in 
 async_pipeline_from_audio_stream
    await pipeline_input.validate()

  File "/usr/src/homeassistant/homeassistant/components/assist_pipeline/pipeline.py", line 1448, in validate
    raise PipelineRunValidationError(

homeassistant.components.assist_pipeline.pipeline.PipelineRunValidationError: the pipeline does not support speech-to-text

WARNING (MainThread) [homeassistant.components.wyoming.satellite] Satellite has been disconnected. Reconnecting in 10 second(s)
betodaviola commented 3 months ago

It was not as straight forward for me, but it finally worked, thanks to @TomasKliner ! Unfortunately, I could not install the add-ons since there is no possibility for me to get a dedicated machine for HA, and add-ons are not supported in the docker container of HA installation. I understand all the arguments of why it is like this and although I agree with most of them, this problem proves that the argument of "but if you install on docker you are good enough to have any add-on on docker as well", which I read around github and reddit posts, is just wrong, but I digress. I installed Piper, Whisper, and openwakeword on docker compose, and then created the pipeline on the assist (the pipeline is an easy process). Finding the compose files that half-worked for my setup, and then trying to find what was wrong and piece something that worked together was pretty difficult, since these compose files are nowhere I can find in the documentation. I don't know why they are not easily available since the container images are already there and documented. If anyone find this thread in a similar situation as me, make sure to use the rhasspy image for the container since there are others, and if anyone wants to see my compose files I will happily share them, just bear in mind that the exact same thing might not work for you. Edited for grammar.