Voice assist using external Wyoming satellite fails post power cycle of the satellite

[jiriappl]

The problem

I have been using https://github.com/synesthesiam/homeassistant-satellite post Year of Voice Chapter 4 and did not encounter this issue. Post Chapter 5, I have reimaged the satellite (rpi3) with the new Wyoming satellite bits https://github.com/rhasspy/wyoming-satellite, and all works initially. In both cases, I have been using wake word detection on the satellite, using wyoming-openwakeword.

However, if I power cycle the rpi satellite, I can see that wake word is detected and audio streaming starts (based on the events that wyoming satellite emits), but on the HA core, the Assist in progress property of the satellite remains off.

Debugging this a bit further, it seems to be that HA/wyoming protocol integration holds an open connection back to the satellite (iirc port 10700), and that connection does not get dropped even after the satellite gets restarted.

If I reboot HA core, or disable and reenable Wyoming integration, my satellite becomes operational as well (though perhaps that needs to happen also post powercycle of the RPi, as trying more iterations of the repro now, I did not get consistent results). So I think the issue is on the Wyoming integration in HA Core, not on the satellite itself.

As for the specific repro steps, my satellite is deployed using the RPI+ReSpeaker instructions from the wyoming-satellite repo and this reproes if I power cycle the RPI satellite. To note, my powercycle period is about 8 hours, not just a short off/on cycle (power off during night).

Re difference between home assistant satellite (chapter 4) and wyoming satellite (chapter 5), I used the token auth post chapter 4 and for chapter 5 I registered the satellite using the wyoming integration. So possibly the connection was established differently between the two.

Would be happy to debug further if you cannot repro or want more details. I spotted the connection from HA Core/Wyoming integration using netstat, executed from an SSH session exposed using the Advanced SSH addons, without protections and tcpdump/netstat from the rpi satellite.

What version of Home Assistant Core has the issue?

core-2024.1.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Wyoming Protocol

Link to integration documentation on our website

https://www.home-assistant.io/integrations/wyoming/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

wyoming-satellite debug log, post reboot (cannot stream to the ha core):

jirka@hamic:~ $ journalctl -fu wyoming-satellite
Jan 13 14:17:16 hamic run[744]: DEBUG:root:Namespace(mic_uri=None, mic_command='arecord -D plughw:CARD=Array,DEV=0 -r 16000 -c 1 -f S16_LE -t raw', mic_command_rate=16000, mic_command_width=2, mic_command_channels=1, mic_command_samples_per_chunk=1024, mic_volume_multiplier=1.0, mic_noise_suppression=0, mic_auto_gain=0, snd_uri=None, snd_command='aplay -D plughw:CARD=Headphones,DEV=0 -r 22050 -c 1 -f S16_LE -t raw', snd_command_rate=22050, snd_command_width=2, snd_command_channels=1, snd_volume_multiplier=1.0, wake_uri='tcp://127.0.0.1:10400', wake_word_name=['ok_nabu', 'ok_nabu'], wake_command=None, wake_command_rate=16000, wake_command_width=2, wake_command_channels=1, vad=False, vad_threshold=0.5, vad_trigger_level=1, vad_buffer_seconds=2, vad_wake_word_timeout=5.0, event_uri='tcp://127.0.0.1:10500', startup_command=None, detect_command=None, detection_command=None, transcript_command=None, stt_start_command=None, stt_stop_command=None, synthesize_command=None, tts_start_command=None, tts_stop_command=None, streaming_start_command=None, streaming_stop_command=None, error_command=None, awake_wav='sounds/done.wav', done_wav='sounds/done.wav', uri='tcp://0.0.0.0:10700', name='hamic', area=None, no_zeroconf=True, zeroconf_name=None, zeroconf_host=None, debug=True)
Jan 13 14:17:16 hamic run[744]: INFO:root:Ready
Jan 13 14:17:16 hamic run[744]: DEBUG:root:Connecting to mic service: ['arecord', '-D', 'plughw:CARD=Array,DEV=0', '-r', '16000', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
Jan 13 14:17:16 hamic run[744]: DEBUG:root:Connecting to snd service: ['aplay', '-D', 'plughw:CARD=Headphones,DEV=0', '-r', '22050', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
Jan 13 14:17:16 hamic run[744]: DEBUG:root:Connecting to wake service: tcp://127.0.0.1:10400
Jan 13 14:17:16 hamic run[744]: DEBUG:root:Connecting to event service: tcp://127.0.0.1:10500
Jan 13 14:17:16 hamic run[744]: INFO:root:Connected to services
Jan 13 14:17:16 hamic run[744]: DEBUG:root:Connected to mic service
Jan 13 14:17:16 hamic run[744]: DEBUG:root:Connected to wake service
Jan 13 14:17:16 hamic run[746]: Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
Jan 13 18:46:51 hamic run[744]: DEBUG:root:Streaming audio
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: run-pipeline
Jan 13 18:46:51 hamic run[744]: DEBUG:root:Connected to snd service
Jan 13 18:46:51 hamic run[744]: DEBUG:root:Connected to event service
Jan 13 18:46:51 hamic run[825]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:51 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:52 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
Jan 13 18:46:52 hamic run[744]: DEBUG:root:event_to_server: audio-chunk
...
and it keeps forever sending DEBUG:root:event_to_server: audio-chunk, it does not transition to the other later states (despite there being no additional voice commands or background sounds). 

### Additional information

_No response_

[home-assistant[bot]]

Hey there @balloob, @synesthesiam, mind taking a look at this issue as it has been labeled with an integration (wyoming) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `wyoming` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign wyoming` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

wyoming documentation wyoming source _{^{(message by IssueLinks)}}

[balloob]

I wonder if we should implement a ping command to detect disconnects.

[atoulmin]

I have this issue as well, once the satellite is rebooted i have to restart HA or the integration to get a connection working

[CoreyJ87]

Im having what I think is this issue? Although, its happening even without reboots. Every couple commands I have to restart the integration and that fixes it for me.

[ragaimeena]

I have the same issue. Any resolution to this?

[synesthesiam]

For the next version of HA (2024.2) and Wyoming satellite version 1.1, this will be be fixed. Pinging between HA and the satellite has been implemented so that the connection can be dropped quickly once either side disconnects.