search

NeonGeckoCom / neon_debos

Debos build files for Neon OS
Apache License 2.0
1 stars 8 forks source link

[BUG] Broken IPv6 on LAN causes hotwords load failure due to timeout #21

Open strugee opened 11 months ago

strugee commented 11 months ago

Description

Having IPv6 configured on the local network, but broken, causes Neon to not respond to the wake word. Digging into the logs shows this:

2023-07-15 02:38:23.773 - voice - ovos_dinkum_listener.service:_connect_to_bus:379 - INFO - Connected to Mycroft Core message bus
2023-07-15 02:38:24.359 - voice - neon_speech.stt:create:81 - INFO - Create STT with config: {'module': 'google_cloud_streaming', 'fallback_module': 'ovos-stt-plugin-vosk', 'ovos-stt-plugin-server': {'url': 'https://stt.openvoiceos.com/stt'}, 'ovos-stt-plugin-vosk': {'model': '/home/neon/.local/share/neon/vosk-model-small-en-us-0.15'}, 'lang': 'en-us'}
2023-07-15 02:38:28.618 - voice - neon_speech.stt:create:81 - INFO - Create STT with config: {'module': 'ovos-stt-plugin-vosk', 'ovos-stt-plugin-vosk': {'model': '/home/neon/.local/share/neon/vosk-model-small-en-us-0.15', 'lang': 'en-us'}}
2023-07-15 02:38:30.865 - voice - neon_utils.signal_utils:init_signal_handlers:107 - INFO - Signal Manager Available
2023-07-15 02:38:40.249 - voice - ovos_plugin_manager.ocp:load:54 - INFO - Loaded OCP plugin: ovos-ocp-m3u-plugin
2023-07-15 02:38:42.680 - voice - ovos_plugin_manager.ocp:load:54 - INFO - Loaded OCP plugin: ovos-ocp-rss-plugin
2023-07-15 02:38:44.752 - voice - ovos_plugin_manager.ocp:load:54 - INFO - Loaded OCP plugin: ovos-ocp-news-plugin
2023-07-15 02:38:46.847 - voice - ovos_plugin_manager.ocp:load:54 - INFO - Loaded OCP plugin: ovos-ocp-files-plugin
2023-07-15 02:38:48.309 - voice - ovos_workshop.skills.layers:<module>:3 - WARNING - Deprecation version=0.1.0. Caller=ovos_workshop.skills.ovos:16. Import from `ovos_workshop.decorators.layers`
2023-07-15 02:38:50.790 - voice - neon_utils.signal_utils:init_signal_handlers:127 - INFO - Overrode mycroft.util.signal methods
2023-07-15 02:38:50.858 - voice - neon_speech.stt:create:81 - INFO - Create STT with config: {'module': 'google_cloud_streaming', 'fallback_module': 'ovos-stt-plugin-vosk', 'ovos-stt-plugin-server': {'url': 'https://stt.openvoiceos.com/stt'}, 'ovos-stt-plugin-vosk': {'model': '/home/neon/.local/share/neon/vosk-model-small-en-us-0.15', 'lang': 'en-us'}}
2023-07-15 02:38:50.970 - voice - ovos_dinkum_listener.service:_before_start:280 - INFO - Starting service...
2023-07-15 02:40:30.474 - voice - neon_speech.service:shutdown:130 - INFO - Shutting Down
2023-07-15 02:41:00.483 - voice - ovos_dinkum_listener.service:_shutdown:359 - ERROR - Shutdown failed with: Timed out waiting for Hotwords load
Traceback (most recent call last):
  File "/home/neon/venv/lib/python3.7/site-packages/ovos_dinkum_listener/service.py", line 352, in _shutdown
    self.hotwords.shutdown()
  File "/home/neon/venv/lib/python3.7/site-packages/ovos_dinkum_listener/voice_loop/hotwords.py", line 330, in shutdown
    for engine in self.plugins:
  File "/home/neon/venv/lib/python3.7/site-packages/ovos_dinkum_listener/voice_loop/hotwords.py", line 79, in wrapped
    raise TimeoutError("Timed out waiting for Hotwords load")
TimeoutError: Timed out waiting for Hotwords load

Steps to Reproduce

  1. Image a USB drive with a fresh Neon Mark II image.
  2. Have IPv6 on the local network, but not globally routable. IIRC (which I might not, this was a number of days ago) my Mark II was getting a link-local address from the router, but nothing else.
  3. Boot the Mark II.
  4. Notice how even when first boot is complete, Neon does not respond to the wake word. Invoking neon-cli and typing queries works fine.

Relevant Code

No response

Other Notes

From my macOS work laptop on the affected network with the default network settings:

% curl -v6 google.com
*   Trying [2607:f8b0:4006:817::200e]:80...
* connect to 2607:f8b0:4006:817::200e port 80 failed: Operation timed out
* Failed to connect to google.com port 80 after 75011 ms: Couldn't connect to server
* Closing connection 0
curl: (28) Failed to connect to google.com port 80 after 75011 ms: Couldn't connect to server

So this appears to not really be a configuration problem on the host side - probably all that can be done is to fall back to IPv4 automatically in case of timeout (or disable IPv6 wholesale).

strugee commented 10 months ago

For those stumbling upon this issue: disabling IPv6 with sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1 will work, but if you go to make this persistent in /etc/sysctl.d, it won't actually disable IPv6 on the interface (though sudo sysctl net.ipv6.conf.all.disable_ipv6 will still report 1). My guess is that when this sysctl is written to, the kernel loops over active network interfaces - but when the variable is applied during early boot, the network hasn't been set up yet.

To fix, you want nmcli connection modify <ssid> ipv6.method disabled followed by a reboot instead. (At least I think - I'm testing now.)