home-assistant / supervisor

:house_with_garden: Home Assistant Supervisor
https://home-assistant.io/hassio/
Apache License 2.0
1.76k stars 638 forks source link

After Supervisor Updates, Can't Restart, Reboot, or Access Add-ons #5353

Open dxmnkd316 opened 2 weeks ago

dxmnkd316 commented 2 weeks ago

Describe the issue you are experiencing

After an automatic Supervisor update, my system is unable to restart or reboot and I cannot access any of the add-ons (file editor, etc.)

I can't reboot even through the CLI. Eventually I need to unplug the Pi and power back up. Which isn't ideal.

I'm wondering if there's something going on with my connection to the external HDD I use for backups. See the CLI errors near the bottom for the exact error.

I have redacted some information below and in the debug file with Xs.

Core Logs:

Error fetching hassio data: Error on Supervisor API: ``` Log details (ERROR) Logger: homeassistant.components.hassio.coordinator Source: helpers/update_coordinator.py:386 integration: Home Assistant Supervisor ([documentation](https://www.home-assistant.io/integrations/hassio), [issues](https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+hassio%22)) First occurred: 11:35:53 AM (1 occurrences) Last logged: 11:35:53 AM ```
Client error on api app/entrypoint.js request Cannot connect to host 172.30.32.2:80 ssl:default [Connect call failed ('172.30.32.2', 80)] ``` Logger: homeassistant.components.hassio.http Source: components/hassio/http.py:190 integration: Home Assistant Supervisor (documentation, issues) First occurred: 12:31:29 PM (5 occurrences) Last logged: 12:34:19 PM Client error on api app/entrypoint.js request Cannot connect to host 172.30.32.2:80 ssl:default [Connect call failed ('172.30.32.2', 80)] Client error on api supervisor/logs request Cannot connect to host 172.30.32.2:80 ssl:default [Connect call failed ('172.30.32.2', 80)] ```
Can't read Supervisor data: ``` Logger: homeassistant.components.hassio Source: components/hassio/__init__.py:448 integration: Home Assistant Supervisor (documentation, issues) First occurred: 11:38:29 AM (13 occurrences) Last logged: 12:38:30 PM Can't read Supervisor data: ```
Client error on /supervisor/info request Cannot connect to host 172.30.32.2:80 ssl:default [Connect call failed ('172.30.32.2', 80)] ``` Logger: homeassistant.components.hassio.handler Source: components/hassio/handler.py:560 integration: Home Assistant Supervisor (documentation, issues) First occurred: 11:35:53 AM (214 occurrences) Last logged: 12:42:55 PM Client error on /supervisor/stats request Cannot connect to host 172.30.32.2:80 ssl:default [Connect call failed ('172.30.32.2', 80)] Client error on /info request Cannot connect to host 172.30.32.2:80 ssl:default [Connect call failed ('172.30.32.2', 80)] Client error on /core/info request Cannot connect to host 172.30.32.2:80 ssl:default [Connect call failed ('172.30.32.2', 80)] Client error on /supervisor/info request Cannot connect to host 172.30.32.2:80 ssl:default [Connect call failed ('172.30.32.2', 80)] Client error on /os/info request Cannot connect to host 172.30.32.2:80 ssl:default [Connect call failed ('172.30.32.2', 80)] ```
Failed to to call /ingress/session ``` Logger: homeassistant.components.hassio Source: components/hassio/websocket_api.py:135 integration: Home Assistant Supervisor (documentation, issues) First occurred: October 12, 2024 at 11:45:36 AM (85 occurrences) Last logged: 12:42:55 PM Failed to to call /addons - Failed to to call /ingress/validate_session - Failed to to call /ingress/session - Failed to to call /supervisor/stats - Failed to to call /resolution/info - ```

System Health: I get the following error in the System Health:
Supervisor API | unreachable

Observer: Going to the Observer page, I get this: Supervisor: | Disconnected

Last logs in Observer are:

2024-10-14 11:33:30.584 INFO (MainThread) [supervisor.misc.tasks] Found new Supervisor version 2024.10.2, updating
2024-10-14 11:33:30.600 INFO (MainThread) [supervisor.supervisor] Fetching AppArmor profile https://ve

Home Assistant CLI: I see the follow lines: CIFS: VFS: \192.168.xxx.xxx has not responded in 180 seconds. Reconnecting... (Note: this is the address of the external USB I have attached to a router used by Samba for external back-ups.) INFO: task python3:2140 blocked for more than 120 seconds.
Tainted: G C 6.6.31-haosraspi # 1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message (this repeats several times) Bluetooth: hci0: Opcode 0x2005 failed: -16 Bluetooth: hci0: Opcode 0x2005 failed: -16

If I try Ctrl+C at the command line it tries to restart Supervisor but fails.

If I try "supervisor restart" or any other command, i get something like: Get "http://supervisor/network/info": dial tcp 172.30.32.2:80: connect: connection refused

What type of installation are you running?

Home Assistant OS

Which operating system are you running on?

Home Assistant Operating System

Steps to reproduce the issue

  1. Supervisor update shows up on update list
  2. It appears that it gets automatically installed
  3. Errors received

Anything in the Supervisor logs that might be useful for us?

I receive the following error when trying to access the logs:

Failed to get supervisor logs, 502: Bad Gateway

System Health information

System Information

version core-2024.10.0
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.12.4
os_name Linux
os_version 6.6.31-haos-raspi
arch aarch64
timezone America/Chicago
config_dir /config
Home Assistant Community Store GitHub API | ok -- | -- GitHub Content | ok GitHub Web | ok HACS Data | ok GitHub API Calls Remaining | 5000 Installed Version | 2.0.1 Stage | running Available Repositories | 1439 Downloaded Repositories | 52
Home Assistant Cloud logged_in | false -- | -- can_reach_cert_server | ok can_reach_cloud_auth | ok can_reach_cloud | ok
Home Assistant Supervisor host_os | Home Assistant OS 13.1 -- | -- update_channel | stable supervisor_version | supervisor-2024.10.0 agent_version | 1.6.0 docker_version | 26.1.4 disk_total | 457.7 GB disk_used | 132.4 GB healthy | true supported | true host_connectivity | true supervisor_connectivity | true ntp_synchronized | true virtualization | board | rpi4-64 supervisor_api | failed to load: unreachable version_api | ok installed_addons | Samba share (12.3.2), File editor (5.8.0), AMR2MQTT (2.1.2), Mosquitto broker (6.4.1), SQLite Web (3.9.2), Terminal & SSH (9.15.0), Samba Backup (5.2.0), InfluxDB (5.0.1), rtl_433 MQTT Auto Discovery (0.8.2), rtl_433 (0.5.2), rtlamr2mqtt (2.3.4), Matter Server (6.6.0), Duck DNS (1.18.0), NGINX Home Assistant SSL proxy (3.11.0), Xcel iTron MQTT (1.2.1), ESPresense Companion (1.0.3), eufy-security-ws (1.9.1), Advanced SSH & Web Terminal (19.0.0), ESPHome (2024.9.2), MQTT Explorer (browser-1.0.3)
Dashboards dashboards | 10 -- | -- resources | 28 views | 82 mode | storage
Recorder oldest_recorder_run | October 4, 2024 at 8:11 PM -- | -- current_recorder_run | October 10, 2024 at 6:36 PM estimated_db_size | 1359.88 MiB database_engine | sqlite database_version | 3.45.3
Spotify api_endpoint_reachable | ok -- | --

Supervisor diagnostics

config_entry-hassio-ce75f0dbdde2562521b1d6bda503879b.json

Additional information

No response

Baeen commented 2 weeks ago

This seemed to be caused by whatever updates the supervisor docker container, doesn't re-assign the right IP. Home Assistant seems to require the supervisor to be at address 172.30.32.2

Please update it with your relevant details.

try this:

docker network ls

NETWORK ID NAME DRIVER SCOPE cb8d8dca6a37 bridge bridge local 00c365f778db hassio bridge local 19195f3a9c2f host host local c2b89c66798d none null local

docker network disconnect hassio hassio_supervisor

docker network connect --ip 172.30.32.2 hassio hassio_supervisor

dxmnkd316 commented 2 weeks ago

Thanks for the reply.

I had originally pulled all IPs out of my big report blindly but realized what 172.30.32.2 was. I added it back into the big report right after posting.

I assume this was meant to run off the CLI. Unless I missed something, docker doesn't have network as a command.

I did check the network info from the CLI and I get the following (summarized):

Docker: Address: 172.30.32.0/23 DNS: 172.30.32.3 Gateway: 172.30.32.1

interface: end0 Connected: true IPv4 Address: [my static IP for the Pi] IPv6 Two addresses are given

Edit: Oh wait, I'm dumb. I see what you meant.

I ran # docker network ls and got a similar list. Just the network IDs were different.

I ran the other two commands and rebooted. Any way to check to see if this solved it? Or just wait and see?

dxmnkd316 commented 1 week ago

Unfortunately, the above did not fix it and I ran into the same error with the automated update to Supervisor 2024.10.3

Here's the latest Observer logs (note that the log cuts out at ".inter" exactly as pasted below):

2024-10-24 15:03:18.106 INFO (MainThread) [supervisor.misc.tasks] Found new Supervisor version 2024.10.3, updating
2024-10-24 15:03:18.108 INFO (MainThread) [supervisor.supervisor] Fetching AppArmor profile https://version.home-assistant.io/apparmor_stable.txt
2024-10-24 15:03:18.252 INFO (MainThread) [supervisor.host.apparmor] Adding/updating AppArmor profile: hassio-supervisor
2024-10-24 15:03:18.565 INFO (MainThread) [supervisor.supervisor] Update Supervisor to version 2024.10.3
2024-10-24 15:03:18.565 INFO (MainThread) [supervisor.docker.inter

All logs are identical to what was listed in the original bug report.