search

home-assistant / operating-system

:beginner: Home Assistant Operating System
Apache License 2.0
4.89k stars 971 forks source link

Part of HA unavailable once a week after upgrading to 7.1 #1723

Closed dallas6672 closed 2 years ago

dallas6672 commented 2 years ago

Describe the issue you are experiencing

I just had my second crash since upgrading to 7.1. Was previously running 5.2 for about a year (which ran great) since I was having the same issue as #1119. I am running off an SD Card, but I do have a USB device connected (Wyze Sense bridge). I enabled the Memory monitor after the first crash. Free memory stayed around 3,347 megs when everything was working. After the crash it increased to about 3,390 megs.

I can still access a few of the menus from the browser like the Dashboards, Maps and Developer Tools. Under Configurations, I can access everything under Settings except for Info. I lose access to all my MQTT devices, but can still control my ZHA devices (Sonoff Zigbee bridge running Tasmota). I also lose access to SSH and the observer.

Here are the only logs that show up under Configuration > Settings > Logs...

Failed to to call /supervisor/available_updates - 8:22:31 PM – (ERROR) Home Assistant Supervisor - message first occurred at 8:18:10 PM and shows up 4 times Client error on /core/info request Cannot connect to host 172.30.32.2:80 ssl:default [Connect call failed ('172.30.32.2', 80)] 8:22:31 PM – (ERROR) Home Assistant Supervisor - message first occurred at 8:06:48 PM and shows up 28 times Error handling request 8:22:22 PM – (ERROR) components/http/static.py - message first occurred at 8:18:13 PM and shows up 8 times Client error on api app/entrypoint.js request Cannot connect to host 172.30.32.2:80 ssl:default [Connect call failed ('172.30.32.2', 80)] 8:22:21 PM – (ERROR) Home Assistant Supervisor - message first occurred at 8:18:13 PM and shows up 2 times Can't read Supervisor data: 8:21:58 PM – (WARNING) Home Assistant Supervisor - message first occurred at 8:06:48 PM and shows up 4 times Error doing job: Task exception was never retrieved 8:21:23 PM – (ERROR) helpers/storage.py - message first occurred at 8:11:23 PM and shows up 4 times Unhandled exception 8:18:39 PM – (ERROR) /usr/local/lib/python3.9/site-packages/aiohttp/web_protocol.py - message first occurred at 8:18:38 PM and shows up 10 times Error fetching openweathermap data: API call timeouted 8:07:52 PM – (ERROR) OpenWeatherMap - message first occurred at January 13, 2022, 12:00:18 AM and shows up 3 times Disconnected from MQTT server core-mosquitto:1883 (16) 8:06:10 PM – (WARNING) MQTT - message first occurred at 8:06:10 PM and shows up 2 times

What operating system image do you use?

rpi4 (Raspberry Pi 4/400 32-bit OS)

What version of Home Assistant Operating System is installed?

7.1

Did you upgrade the Operating System.

Yes

Steps to reproduce the issue

  1. Allow HA to run for about a week and eventually it crashes

Anything in the Supervisor logs that might be useful for us?

Unable to access once it's crashed

Anything in the Host logs that might be useful for us?

Unable to access once it's crashed

System Health information

System Health

version core-2021.12.9
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.9.7
os_name Linux
os_version 5.10.63-v7l
arch armv7l
timezone America/New_York
Home Assistant Supervisor host_os | Home Assistant OS 7.1 -- | -- update_channel | stable supervisor_version | supervisor-2021.12.2 docker_version | 20.10.9 disk_total | 56.6 GB disk_used | 5.6 GB healthy | true supported | true board | rpi4 supervisor_api | ok version_api | ok installed_addons | Terminal & SSH (9.3.0), Glances (0.14.1), Mosquitto broker (6.0.1), ESPHome (2021.12.3), TasmoAdmin (0.16.0), TasmoBackup (1.04.26)
Lovelace dashboards | 2 -- | -- resources | 1 views | 6 mode | storage

Additional information

Raspberry Pi 4 Model B Rev 1.2

agners commented 2 years ago

I can still access a few of the menus from the browser like the Dashboards, Maps and Developer Tools.

So to be clear crash means that only some part of the web frontend are unavailable? So this is unlikely the OS which crashes, but "only" some services then.

I can still access a few of the menus from the browser like the Dashboards, Maps and Developer Tools.

Is that also true when forcing a page refresh (Ctrl+F5)

Here are the only logs that show up under Configuration > Settings > Logs...

Are those log when the system is partially unavailable?

dallas6672 commented 2 years ago

So to be clear crash means that only some part of the web frontend are unavailable? So this is unlikely the OS which crashes, but "only" some services then.

Correct

I can still access a few of the menus from the browser like the Dashboards, Maps and Developer Tools.

Is that also true when forcing a page refresh (Ctrl+F5)

Don't recall if I tried that or not. I'll be sure test next time it happens.

Are those log when the system is partially unavailable?

Correct

agners commented 2 years ago

I'd also recommend to enable SSH access on OS level using the method described here. This should allow you to connect to the system on OS level next time it goes partly down. From the OS shell, the logs of Supervisor would be particularly interesting. You can get the log using:

docker logs hassio_supervisor
dallas6672 commented 2 years ago

Just crashed again (almost exactly 7 days. crashed last Sunday night), but for some reason I get "connection refused" now when trying to access SSH at the OS level. I set it up using the method you mentioned and it was working earlier in the week. When trying to access SSH through the add-on, the connection just times out as expected.

Tested page refresh (Ctrl+F5) and the only thing that seemed to change was the WebRTC Lovelace card stopped working (Custom element doesn't exist: webrtc-camera).

Basically the same logs as before from Configuration > Settings > Logs...

Error doing job: Task exception was never retrieved 7:54:45 PM – (ERROR) helpers/storage.py - message first occurred at 7:49:35 PM and shows up 3 times

Can't read Supervisor data: 7:53:46 PM – (WARNING) Home Assistant Supervisor

Client error on /host/info request Cannot connect to host 172.30.32.2:80 ssl:default [Connect call failed ('172.30.32.2', 80)] 7:53:46 PM – (ERROR) Home Assistant Supervisor - message first occurred at 7:53:46 PM and shows up 6 times

Error fetching openweathermap data: HTTPSConnectionPool(host='api.openweathermap.org', port=443): Max retries exceeded with url: /data/2.5/weather?APPID=4fe9b26f9fdbf16b99777702fbfda5e5&lang=en&lon=-86.032356&lat=37.72696 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0xa7847598>: Failed to establish a new connection: [Errno -3] Try again')) 7:53:24 PM – (ERROR) OpenWeatherMap

Disconnected from MQTT server core-mosquitto:1883 (16) 7:50:37 PM – (WARNING) MQTT - message first occurred at 7:50:37 PM and shows up 2 times

dallas6672 commented 2 years ago

After rebooting, these are the logs from "docker logs hassio_supervisor" shortly before crashing and the boot after the crash. Crash must of happened shortly after timestamp 22-01-23 19:20:32...

22-01-23 17:13:28 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING
22-01-23 17:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.CORE
22-01-23 17:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.SUPERVISOR
22-01-23 17:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM
22-01-23 17:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON
22-01-23 17:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.PLUGIN
22-01-23 17:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
22-01-23 17:13:28 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-01-23 17:13:28 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
22-01-23 17:13:28 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-01-23 17:13:28 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
22-01-23 17:13:28 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete
22-01-23 17:18:26 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-01-23 17:48:27 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-01-23 18:13:28 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING
22-01-23 18:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.CORE
22-01-23 18:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.SUPERVISOR
22-01-23 18:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM
22-01-23 18:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON
22-01-23 18:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.PLUGIN
22-01-23 18:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
22-01-23 18:13:28 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-01-23 18:13:28 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
22-01-23 18:13:28 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-01-23 18:13:28 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
22-01-23 18:13:28 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete
22-01-23 18:14:37 INFO (MainThread) [supervisor.updater] Fetching update data from https://version.home-assistant.io/stable.json
22-01-23 18:18:27 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-01-23 18:48:28 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-01-23 18:56:43 INFO (MainThread) [supervisor.host.info] Updating local host information
22-01-23 18:56:44 INFO (MainThread) [supervisor.host.services] Updating service information
22-01-23 18:56:45 INFO (MainThread) [supervisor.host.network] Updating local network information
22-01-23 18:56:45 INFO (MainThread) [supervisor.host.sound] Updating PulseAudio information
22-01-23 18:56:45 INFO (MainThread) [supervisor.host.manager] Host information reload completed
22-01-23 19:13:28 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING
22-01-23 19:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.CORE
22-01-23 19:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.SUPERVISOR
22-01-23 19:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM
22-01-23 19:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON
22-01-23 19:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.PLUGIN
22-01-23 19:13:28 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
22-01-23 19:13:28 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-01-23 19:13:28 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
22-01-23 19:13:28 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-01-23 19:13:28 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
22-01-23 19:13:28 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete
22-01-23 19:18:28 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-01-23 19:20:25 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/danmed/TasmoBackupV1 repository
22-01-23 19:20:25 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/sjthespian/addon-tuyagateway repository
22-01-23 19:20:25 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/home-assistant/addons repository
22-01-23 19:20:25 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/hassio-addons/repository repository
22-01-23 19:20:32 INFO (MainThread) [supervisor.store] Loading add-ons from store: 64 all - 0 new - 0 remove
22-01-23 19:20:32 INFO (MainThread) [supervisor.store] Loading add-ons from store: 64 all - 0 new - 0 remove
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] udev.sh: executing...
[03:41:33] INFO: Using udev information from host
[cont-init.d] udev.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[03:41:34] INFO: Starting local supervisor watchdog...
22-01-24 03:41:37 INFO (MainThread) [__main__] Initializing Supervisor setup
22-01-24 03:41:37 INFO (MainThread) [supervisor.bootstrap] Initializing Supervisor Sentry
22-01-23 22:41:37 INFO (MainThread) [supervisor.bootstrap] Seting up coresys for machine: raspberrypi4
22-01-23 22:41:37 INFO (SyncWorker_0) [supervisor.docker.supervisor] Attaching to Supervisor homeassistant/armv7-hassio-supervisor with version 2021.12.2
22-01-23 22:41:38 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.INITIALIZE
22-01-23 22:41:38 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-01-23 22:41:38 INFO (MainThread) [__main__] Setting up Supervisor
22-01-23 22:41:38 INFO (MainThread) [supervisor.api] Starting API on 172.30.32.2
22-01-23 22:41:38 INFO (MainThread) [supervisor.hardware.monitor] Started Supervisor hardware monitor
22-01-23 22:41:38 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface io.hass.os
22-01-23 22:41:38 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.systemd1
22-01-23 22:41:38 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.login1
22-01-23 22:41:38 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.hostname1
22-01-23 22:41:39 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.timedate1
22-01-23 22:41:39 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.NetworkManager
22-01-23 22:41:39 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface de.pengutronix.rauc
22-01-23 22:41:39 INFO (MainThread) [supervisor.host.info] Updating local host information
22-01-23 22:41:39 INFO (MainThread) [supervisor.host.services] Updating service information
22-01-23 22:41:39 INFO (MainThread) [supervisor.host.network] Updating local network information
22-01-23 22:41:40 INFO (MainThread) [supervisor.host.sound] Updating PulseAudio information
22-01-23 22:41:40 INFO (MainThread) [supervisor.host.manager] Host information reload completed
22-01-23 22:41:40 INFO (MainThread) [supervisor.host.apparmor] Loading AppArmor Profiles: {'hassio-supervisor'}
22-01-23 22:41:40 INFO (SyncWorker_0) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/armv7-hassio-cli with version 2021.09.0
22-01-23 22:41:40 INFO (MainThread) [supervisor.plugins.cli] Starting CLI plugin
22-01-23 22:41:40 INFO (SyncWorker_0) [supervisor.docker.interface] Cleaning hassio_cli application
22-01-23 22:41:42 INFO (SyncWorker_0) [supervisor.docker.cli] Starting CLI ghcr.io/home-assistant/armv7-hassio-cli with version 2021.09.0 - 172.30.32.5
22-01-23 22:41:42 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/armv7-hassio-dns with version 2021.06.0
22-01-23 22:41:42 INFO (MainThread) [supervisor.plugins.dns] Starting CoreDNS plugin
22-01-23 22:41:42 INFO (SyncWorker_1) [supervisor.docker.interface] Cleaning hassio_dns application
22-01-23 22:41:44 INFO (SyncWorker_1) [supervisor.docker.dns] Starting DNS ghcr.io/home-assistant/armv7-hassio-dns with version 2021.06.0 - 172.30.32.3
22-01-23 22:41:44 INFO (MainThread) [supervisor.plugins.dns] Updated /etc/resolv.conf
22-01-23 22:41:44 INFO (SyncWorker_0) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/armv7-hassio-audio with version 2021.07.0
22-01-23 22:41:44 INFO (MainThread) [supervisor.plugins.audio] Starting Audio plugin
22-01-23 22:41:44 INFO (SyncWorker_0) [supervisor.docker.interface] Cleaning hassio_audio application
22-01-23 22:41:47 INFO (SyncWorker_0) [supervisor.docker.audio] Starting Audio ghcr.io/home-assistant/armv7-hassio-audio with version 2021.07.0 - 172.30.32.4
22-01-23 22:41:47 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/armv7-hassio-observer with version 2021.10.0
22-01-23 22:41:47 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to homeassistant/armv7-hassio-multicast with version 2021.04.0
22-01-23 22:41:47 INFO (MainThread) [supervisor.plugins.multicast] Starting Multicast plugin
22-01-23 22:41:47 INFO (SyncWorker_1) [supervisor.docker.interface] Cleaning hassio_multicast application
22-01-23 22:41:48 INFO (SyncWorker_1) [supervisor.docker.multicast] Starting Multicast homeassistant/armv7-hassio-multicast with version 2021.04.0 - Host
22-01-23 22:41:48 INFO (MainThread) [supervisor.updater] Fetching update data from https://version.home-assistant.io/stable.json
22-01-23 22:41:49 INFO (SyncWorker_0) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/raspberrypi4-homeassistant with version 2021.12.9
22-01-23 22:41:49 INFO (MainThread) [supervisor.homeassistant.secrets] Loaded 7 Home Assistant secrets
22-01-23 22:41:49 INFO (MainThread) [supervisor.os.manager] Detect Home Assistant Operating System 7.1 / BootSlot A
22-01-23 22:41:52 INFO (MainThread) [supervisor.store.git] Loading add-on /data/addons/git/9409ee68 repository
22-01-23 22:41:52 INFO (MainThread) [supervisor.store.git] Loading add-on /data/addons/git/ba9f8461 repository
22-01-23 22:41:52 INFO (MainThread) [supervisor.store.git] Loading add-on /data/addons/core repository
22-01-23 22:41:52 INFO (MainThread) [supervisor.store.git] Loading add-on /data/addons/git/a0d7b954 repository
22-01-23 22:41:56 INFO (MainThread) [supervisor.store] Loading add-ons from store: 64 all - 64 new - 0 remove
22-01-23 22:41:56 INFO (MainThread) [supervisor.addons] Found 6 installed add-ons
22-01-23 22:41:56 INFO (SyncWorker_3) [supervisor.docker.interface] Attaching to homeassistant/armv7-addon-ssh with version 9.3.0
22-01-23 22:41:56 INFO (SyncWorker_2) [supervisor.docker.interface] Attaching to homeassistant/armv7-addon-mosquitto with version 6.0.1
22-01-23 22:41:56 INFO (SyncWorker_4) [supervisor.docker.interface] Attaching to ghcr.io/hassio-addons/glances/armv7 with version 0.14.1
22-01-23 22:41:56 INFO (SyncWorker_5) [supervisor.docker.interface] Attaching to esphome/esphome-hassio-armv7 with version 2022.1.1
22-01-23 22:41:56 INFO (SyncWorker_0) [supervisor.docker.interface] Attaching to ghcr.io/hassio-addons/tasmoadmin/armv7 with version 0.16.0
22-01-23 22:41:56 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to danmed/tasmobackupv1 with version 1.04.26
22-01-23 22:41:56 INFO (MainThread) [supervisor.backups.manager] Found 44 backup files
22-01-23 22:41:56 INFO (MainThread) [supervisor.discovery] Loaded 1 messages
22-01-23 22:41:56 INFO (MainThread) [supervisor.ingress] Loaded 0 ingress sessions
22-01-23 22:41:56 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.SETUP
22-01-23 22:41:56 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-01-23 22:41:56 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.SETUP
22-01-23 22:41:57 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-01-23 22:41:57 INFO (MainThread) [supervisor.jobs] 'ResolutionFixup.run_autofix' blocked from execution, system is not running - CoreState.SETUP
22-01-23 22:41:57 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.SETUP
22-01-23 22:41:57 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-01-23 22:41:57 INFO (MainThread) [__main__] Running Supervisor
22-01-23 22:41:57 INFO (MainThread) [supervisor.os.manager] Rauc: A - marked slot kernel.0 as good
22-01-23 22:41:57 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.INITIALIZE' starting 0 add-ons
22-01-23 22:41:57 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.SYSTEM' starting 1 add-ons
22-01-23 22:41:57 WARNING (MainThread) [supervisor.addons.options] Option 'anonymous' does not exist in the schema for Mosquitto broker (core_mosquitto)
22-01-23 22:41:57 INFO (SyncWorker_1) [supervisor.docker.interface] Cleaning addon_core_mosquitto application
22-01-23 22:41:57 WARNING (SyncWorker_1) [supervisor.addons.options] Option 'anonymous' does not exist in the schema for Mosquitto broker (core_mosquitto)
22-01-23 22:42:00 INFO (SyncWorker_1) [supervisor.docker.addon] Starting Docker add-on homeassistant/armv7-addon-mosquitto with version 6.0.1
22-01-23 22:42:04 INFO (MainThread) [supervisor.services.modules.mqtt] Set core_mosquitto as service provider for mqtt
22-01-23 22:42:05 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.SERVICES' starting 1 add-ons
22-01-23 22:42:05 INFO (SyncWorker_3) [supervisor.docker.interface] Cleaning addon_core_ssh application
22-01-23 22:42:08 INFO (SyncWorker_3) [supervisor.docker.addon] Starting Docker add-on homeassistant/armv7-addon-ssh with version 9.3.0
22-01-23 22:42:13 INFO (MainThread) [supervisor.core] Start Home Assistant Core
22-01-23 22:42:13 INFO (SyncWorker_1) [supervisor.docker.interface] Starting homeassistant
22-01-23 22:42:13 INFO (MainThread) [supervisor.homeassistant.core] Wait until Home Assistant is ready
22-01-23 22:42:25 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.STARTUP
22-01-23 22:42:25 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-01-23 22:42:32 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-01-23 22:42:59 WARNING (MainThread) [supervisor.addons.options] Option 'anonymous' does not exist in the schema for Mosquitto broker (core_mosquitto)
22-01-23 22:43:03 INFO (MainThread) [supervisor.homeassistant.core] Detect a running Home Assistant instance
22-01-23 22:43:03 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.APPLICATION' starting 0 add-ons
22-01-23 22:43:03 INFO (MainThread) [supervisor.misc.tasks] All core tasks are scheduled
22-01-23 22:43:03 INFO (MainThread) [supervisor.core] Supervisor is up and running
22-01-23 22:43:03 INFO (MainThread) [supervisor.host.info] Updating local host information
22-01-23 22:43:03 INFO (MainThread) [supervisor.updater] Fetching update data from https://version.home-assistant.io/stable.json
22-01-23 22:43:03 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING
22-01-23 22:43:03 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.CORE
22-01-23 22:43:03 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.SUPERVISOR
22-01-23 22:43:03 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM
22-01-23 22:43:03 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON
22-01-23 22:43:03 WARNING (MainThread) [supervisor.utils.pwned] Can't fetch HIBP data: Cannot connect to host api.pwnedpasswords.com:443 ssl:default [Connect call failed ('0.0.0.0', 443)]
22-01-23 22:43:03 WARNING (MainThread) [supervisor.addons.options] Option 'anonymous' does not exist in the schema for Mosquitto broker (core_mosquitto)
22-01-23 22:43:03 WARNING (MainThread) [supervisor.utils.pwned] Can't fetch HIBP data: Cannot connect to host api.pwnedpasswords.com:443 ssl:default [Connect call failed ('0.0.0.0', 443)]
22-01-23 22:43:03 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.PLUGIN
22-01-23 22:43:03 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
22-01-23 22:43:03 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-01-23 22:43:03 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
22-01-23 22:43:03 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-01-23 22:43:03 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
22-01-23 22:43:03 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete
22-01-23 22:43:05 INFO (MainThread) [supervisor.host.services] Updating service information
22-01-23 22:43:05 INFO (MainThread) [supervisor.host.network] Updating local network information
22-01-23 22:43:05 INFO (MainThread) [supervisor.host.sound] Updating PulseAudio information
22-01-23 22:43:05 INFO (MainThread) [supervisor.host.manager] Host information reload completed
agners commented 2 years ago

Hm, it seems as is the Supervisor just disappeared, without even shutting down. That is strange.

Can you check all the logs from the time window around there, e.g. using this command:

journalctl --since "22-01-23 19:20:00" --until "22-01-23 19:30:00"

(you might want to adjust/extend the time range, depending on what shows up in the log exactly.

dallas6672 commented 2 years ago

journalctl --since "22-01-23 19:20:00" --until "22-01-23 19:30:00"

-- Journal begins at Fri 2021-04-23 09:22:20 UTC, ends at Mon 2022-01-24 13:24:47 UTC. --
Jan 23 19:20:45 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]:
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965359: New client connected from 192.168.4.148 as DVES_F6XXXX (p2, c1, k30, u'addons').
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965423: Client <unknown> has exceeded timeout, disconnecting.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965434: Client DVES_F6XXXX has exceeded timeout, disconnecting.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965435: New connection from 192.168.4.148 on port 1883.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965463: New connection from 192.168.4.148 on port 1883.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965463: New client connected from 192.168.4.148 as DVES_F6XXXX (p2, c1, k30, u'addons').
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965480: Client DVES_F6XXXX already connected, closing old connection.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965480: New client connected from 192.168.4.148 as DVES_F6XXXX (p2, c1, k30, u'addons').
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965480: Socket error on client DVES_F6XXXX, disconnecting.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965480: New connection from 192.168.4.148 on port 1883.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965480: New client connected from 192.168.4.148 as DVES_F6XXXX (p2, c1, k30, u'addons').
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965496: Saving in-memory database to /data/mosquitto.db.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965867: New connection from 192.168.4.170 on port 1883.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965867: Client DVES_02XXXX already connected, closing old connection.
Jan 23 19:25:45 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]:

I then told it to show out to 19:50 and got this...

journalctl --since "22-01-23 19:20:00" --until "22-01-23 19:50:00"

-- Journal begins at Fri 2021-04-23 09:22:20 UTC, ends at Mon 2022-01-24 14:07:06 UTC. --
Jan 23 19:20:45 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]:
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965359: New client connected from 192.168.4.148 as DVES_F6XXXX (p2, c1, k30, u'addons').
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965423: Client <unknown> has exceeded timeout, disconnecting.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965434: Client DVES_F6XXXX has exceeded timeout, disconnecting.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965435: New connection from 192.168.4.148 on port 1883.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965463: New connection from 192.168.4.148 on port 1883.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965463: New client connected from 192.168.4.148 as DVES_F6XXXX (p2, c1, k30, u'addons').
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965480: Client DVES_F6XXXX already connected, closing old connection.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965480: New client connected from 192.168.4.148 as DVES_F6XXXX (p2, c1, k30, u'addons').
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965480: Socket error on client DVES_F6XXXX, disconnecting.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965480: New connection from 192.168.4.148 on port 1883.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965480: New client connected from 192.168.4.148 as DVES_F6XXXX (p2, c1, k30, u'addons').
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965496: Saving in-memory database to /data/mosquitto.db.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965867: New connection from 192.168.4.170 on port 1883.
Jan 23 19:24:27 HomeAssistant homeassistant/armv7-addon-mosquitto:6.0.1/addon_core_mosquitto[453]: 1642965867: Client DVES_02XXXX already connected, closing old connection.
Jan 23 19:25:45 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]:
Jan 23 19:30:46 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]:
Jan 23 19:35:46 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]:
Jan 23 19:40:46 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]:
Jan 23 19:43:18 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]: 22-01-23 14:43:18 INFO (MainThread) [supervisor.host.info] Updating local host information
Jan 23 19:43:18 HomeAssistant dbus-daemon[109]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.23' (uid=0 pid=1633 comm="python3 -m supervisor ")
Jan 23 19:43:18 HomeAssistant audit: BPF prog-id=319 op=LOAD
Jan 23 19:43:18 HomeAssistant audit: BPF prog-id=320 op=LOAD
Jan 23 19:43:18 HomeAssistant systemd[1]: Starting Hostname Service...
Jan 23 19:43:18 HomeAssistant kernel: audit: type=1334 audit(1642966998.736:810): prog-id=319 op=LOAD
Jan 23 19:43:18 HomeAssistant kernel: audit: type=1334 audit(1642966998.736:811): prog-id=320 op=LOAD
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/systemd/timesyncd.conf supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/modules-load.d supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/modprobe.d supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/udev/rules.d supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/NetworkManager/system-connections supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/dropbear supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/hosts supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/docker supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/hostname supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/dropbear supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/docker supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/udev/rules.d supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/hosts supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/modules-load.d supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/modprobe.d supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/hostname supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/systemd/timesyncd.conf supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/NetworkManager/system-connections supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant dbus-daemon[109]: [system] Successfully activated service 'org.freedesktop.hostname1'
Jan 23 19:43:19 HomeAssistant systemd[1]: Started Hostname Service.
Jan 23 19:43:19 HomeAssistant dbus-daemon[109]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.25' (uid=0 pid=1633 comm="python3 -m supervisor ")
Jan 23 19:43:19 HomeAssistant audit: BPF prog-id=321 op=LOAD
Jan 23 19:43:19 HomeAssistant audit: BPF prog-id=322 op=LOAD
Jan 23 19:43:19 HomeAssistant systemd[1]: Starting Time & Date Service...
Jan 23 19:43:19 HomeAssistant kernel: audit: type=1334 audit(1642966999.396:812): prog-id=321 op=LOAD
Jan 23 19:43:19 HomeAssistant kernel: audit: type=1334 audit(1642966999.396:813): prog-id=322 op=LOAD
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/docker supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/hostname supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/dropbear supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/modprobe.d supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/hosts supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/NetworkManager/system-connections supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/modules-load.d supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/systemd/timesyncd.conf supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/udev/rules.d supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/docker supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/modules-load.d supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/systemd/timesyncd.conf supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/NetworkManager/system-connections supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/dropbear supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/modprobe.d supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/hosts supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/udev/rules.d supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:19 HomeAssistant kernel: ext4 filesystem being remounted at /run/systemd/unit-root/etc/hostname supports timestamps until 2038 (0x7fffffff)
Jan 23 19:43:20 HomeAssistant dbus-daemon[109]: [system] Successfully activated service 'org.freedesktop.timedate1'
Jan 23 19:43:20 HomeAssistant systemd[1]: Started Time & Date Service.
Jan 23 19:43:20 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]: 22-01-23 14:43:20 INFO (MainThread) [supervisor.host.services] Updating service information
Jan 23 19:43:20 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]: 22-01-23 14:43:20 INFO (MainThread) [supervisor.host.network] Updating local network information
Jan 23 19:43:21 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]: 22-01-23 14:43:21 INFO (MainThread) [supervisor.host.sound] Updating PulseAudio information
Jan 23 19:43:21 HomeAssistant ghcr.io/home-assistant/armv7-hassio-audio:2021.07.0/hassio_audio[453]: I: [pulseaudio] client.c: Created 76 "Native client (UNIX socket client)"
Jan 23 19:43:21 HomeAssistant ghcr.io/home-assistant/armv7-hassio-audio:2021.07.0/hassio_audio[453]: I: [pulseaudio] protocol-native.c: Client authenticated anonymously.
Jan 23 19:43:21 HomeAssistant ghcr.io/home-assistant/armv7-hassio-audio:2021.07.0/hassio_audio[453]: D: [pulseaudio] protocol-native.c: Protocol version: remote 34, local 34
Jan 23 19:43:21 HomeAssistant ghcr.io/home-assistant/armv7-hassio-audio:2021.07.0/hassio_audio[453]: D: [pulseaudio] protocol-native.c: SHM possible: no
Jan 23 19:43:21 HomeAssistant ghcr.io/home-assistant/armv7-hassio-audio:2021.07.0/hassio_audio[453]: D: [pulseaudio] protocol-native.c: Negotiated SHM: no
Jan 23 19:43:21 HomeAssistant ghcr.io/home-assistant/armv7-hassio-audio:2021.07.0/hassio_audio[453]: D: [pulseaudio] protocol-native.c: Disabling srbchannel, reason: No SHM support
Jan 23 19:43:21 HomeAssistant ghcr.io/home-assistant/armv7-hassio-audio:2021.07.0/hassio_audio[453]: I: [pulseaudio] client.c: Freed 76 "supervisor"
Jan 23 19:43:21 HomeAssistant ghcr.io/home-assistant/armv7-hassio-audio:2021.07.0/hassio_audio[453]: I: [pulseaudio] protocol-native.c: Connection died.
Jan 23 19:43:21 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]: 22-01-23 14:43:21 INFO (MainThread) [supervisor.host.manager] Host information reload completed
Jan 23 19:43:49 HomeAssistant systemd[1]: systemd-hostnamed.service: Deactivated successfully.
Jan 23 19:43:49 HomeAssistant audit: BPF prog-id=320 op=UNLOAD
Jan 23 19:43:49 HomeAssistant audit: BPF prog-id=319 op=UNLOAD
Jan 23 19:43:49 HomeAssistant kernel: audit: type=1334 audit(1642967029.466:814): prog-id=320 op=UNLOAD
Jan 23 19:43:49 HomeAssistant kernel: audit: type=1334 audit(1642967029.466:815): prog-id=319 op=UNLOAD
Jan 23 19:43:50 HomeAssistant systemd[1]: systemd-timedated.service: Deactivated successfully.
Jan 23 19:43:50 HomeAssistant audit: BPF prog-id=322 op=UNLOAD
Jan 23 19:43:50 HomeAssistant audit: BPF prog-id=321 op=UNLOAD
Jan 23 19:43:50 HomeAssistant kernel: audit: type=1334 audit(1642967030.106:816): prog-id=322 op=UNLOAD
Jan 23 19:43:50 HomeAssistant kernel: audit: type=1334 audit(1642967030.106:817): prog-id=321 op=UNLOAD
Jan 23 19:45:46 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]:
Jan 23 19:48:20 HomeAssistant homeassistant/armv7-hassio-supervisor:latest/hassio_supervisor[453]: 22-01-23 14:48:20 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
agners commented 2 years ago

It seems the Supervisor is up and doing well. The logs show no indication that something is wrong.

So at ~19:50, the system was not available?

Since you claim its caused by the OS upgrade, can you try a OS downgrade to see if it indeed disappears?

dallas6672 commented 2 years ago

Correct. At 19:50 I was having the same issue listed in the original post. This is the same issue I had over a year ago when I found #1119. That is when I downgraded to 5.2 and the issue went away.

I will downgrade it back to 5.2 unless there's another version you'd like me to try.

dallas6672 commented 2 years ago

A little over a week now and it hasn't crashed yet on 5.2.

dallas6672 commented 2 years ago

Is there anything else you'd like for me to try?

agners commented 2 years ago

I don't really have a clue from the logs what could be happening. Either the logs are incomplete, or the anomaly is not caused by the Raspberry Pi itself.

Maybe its network related somehow?

Have you tried with another SD card? Also a clean install of OS 7.1 on a new SD card running on the same hardware would be interesting.

How do you downgrade? We use the A/B system, so it could be that 7.1 is on bad part of the SD card (wild guess).

dallas6672 commented 2 years ago

Haven't tried wiping the SD card and starting over yet but when I downgrade I just use the "ha os update --version x.x" command

dallas6672 commented 2 years ago

Wiped/Reimaged the SD card Friday (Feb. 11th) and it just crashed again. Also switched to 64 bit since it's now the recommended version. Was 32-bit when I originally installed it.

Weird that it keeps happening on Sunday nights around this time. 7-8 PM.

SSH still refuses the connection. I was able to access the "Info" section this time (See below - supervisor_api | failed to load: unreachable). In the past it would always error out/fail to load the page.

I will try a different SD card now

System Health

version core-2022.2.6
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.9.7
os_name Linux
os_version 5.10.92-v8
arch aarch64
timezone America/New_York
Home Assistant Supervisor host_os | Home Assistant OS 7.3 -- | -- update_channel | stable supervisor_version | supervisor-2022.01.1 docker_version | 20.10.9 disk_total | 56.6 GB disk_used | 3.0 GB healthy | true supported | true board | rpi4-64 supervisor_api | failed to load: unreachable version_api | pending installed_addons | ESPHome (2022.1.4), Glances (0.14.2), Mosquitto broker (6.0.1), TasmoAdmin (0.16.0), TasmoBackup (1.04.26), Terminal & SSH (9.3.0)
Lovelace dashboards | 2 -- | -- resources | 1 views | 6 mode | storage

"docker logs hassio_supervisor" Logs shortly before issue and after the reboot...

22-02-13 16:54:41 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-02-13 16:54:41 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
22-02-13 16:54:41 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-02-13 16:54:41 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
22-02-13 16:54:41 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete
22-02-13 16:54:44 INFO (MainThread) [supervisor.updater] Fetching update data from https://version.home-assistant.io/stable.json
22-02-13 17:15:12 INFO (MainThread) [supervisor.host.info] Updating local host information
22-02-13 17:15:13 INFO (MainThread) [supervisor.host.services] Updating service information
22-02-13 17:15:13 INFO (MainThread) [supervisor.host.network] Updating local network information
22-02-13 17:15:14 INFO (MainThread) [supervisor.host.sound] Updating PulseAudio information
22-02-13 17:15:14 INFO (MainThread) [supervisor.host.manager] Host information reload completed
22-02-13 17:21:23 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-02-13 17:51:23 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-02-13 17:54:41 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING
22-02-13 17:54:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
22-02-13 17:54:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.CORE
22-02-13 17:54:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON
22-02-13 17:54:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.PLUGIN
22-02-13 17:54:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.SUPERVISOR
22-02-13 17:54:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM
22-02-13 17:54:41 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-02-13 17:54:41 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
22-02-13 17:54:41 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-02-13 17:54:41 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
22-02-13 17:54:41 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete
22-02-13 17:55:07 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/danmed/TasmoBackupV1 repository
22-02-13 17:55:07 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/hassio-addons/repository repository
22-02-13 17:55:07 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/home-assistant/addons repository
22-02-13 17:55:07 INFO (MainThread) [supervisor.store.git] Update add-on https://github.com/sjthespian/addon-tuyagateway repository
22-02-13 17:55:10 INFO (MainThread) [supervisor.store] Loading add-ons from store: 64 all - 0 new - 0 remove
22-02-13 17:55:10 INFO (MainThread) [supervisor.store] Loading add-ons from store: 64 all - 0 new - 0 remove
22-02-13 18:21:24 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-02-13 18:51:25 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-02-13 18:54:41 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING
22-02-13 18:54:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
22-02-13 18:54:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.CORE
22-02-13 18:54:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON
22-02-13 18:54:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.PLUGIN
22-02-13 18:54:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.SUPERVISOR
22-02-13 18:54:41 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM
22-02-13 18:54:41 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-02-13 18:54:41 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
22-02-13 18:54:42 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-02-13 18:54:42 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
22-02-13 18:54:42 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete
22-02-13 18:54:44 INFO (MainThread) [supervisor.updater] Fetching update data from https://version.home-assistant.io/stable.json
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] udev.sh: executing...
[00:32:15] INFO: Using udev information from host
[cont-init.d] udev.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[00:32:16] INFO: Starting local supervisor watchdog...
22-02-14 00:32:20 INFO (MainThread) [__main__] Initializing Supervisor setup
22-02-14 00:32:20 INFO (MainThread) [supervisor.bootstrap] Initializing Supervisor Sentry
22-02-13 19:32:20 INFO (MainThread) [supervisor.bootstrap] Seting up coresys for machine: raspberrypi4-64
22-02-13 19:32:20 INFO (SyncWorker_0) [supervisor.docker.supervisor] Attaching to Supervisor homeassistant/aarch64-hassio-supervisor with version 2022.01.1
22-02-13 19:32:20 WARNING (SyncWorker_1) [supervisor.docker.interface] Can't find homeassistant/aarch64-hassio-supervisor for cleanup
22-02-13 19:32:20 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.INITIALIZE
22-02-13 19:32:20 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-02-13 19:32:20 INFO (MainThread) [__main__] Setting up Supervisor
22-02-13 19:32:20 INFO (MainThread) [supervisor.api] Starting API on 172.30.32.2
22-02-13 19:32:20 INFO (MainThread) [supervisor.hardware.monitor] Started Supervisor hardware monitor
22-02-13 19:32:20 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface io.hass.os
22-02-13 19:32:20 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.systemd1
22-02-13 19:32:20 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.login1
22-02-13 19:32:20 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.hostname1
22-02-13 19:32:21 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.timedate1
22-02-13 19:32:21 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface org.freedesktop.NetworkManager
22-02-13 19:32:21 INFO (MainThread) [supervisor.dbus.manager] Load dbus interface de.pengutronix.rauc
22-02-13 19:32:21 INFO (MainThread) [supervisor.host.info] Updating local host information
22-02-13 19:32:21 INFO (MainThread) [supervisor.host.services] Updating service information
22-02-13 19:32:21 INFO (MainThread) [supervisor.host.network] Updating local network information
22-02-13 19:32:22 INFO (MainThread) [supervisor.host.sound] Updating PulseAudio information
22-02-13 19:32:22 INFO (MainThread) [supervisor.host.manager] Host information reload completed
22-02-13 19:32:22 INFO (MainThread) [supervisor.host.apparmor] Loading AppArmor Profiles: {'hassio-supervisor'}
22-02-13 19:32:22 INFO (SyncWorker_0) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/aarch64-hassio-cli with version 2021.12.0
22-02-13 19:32:22 INFO (MainThread) [supervisor.plugins.cli] Starting CLI plugin
22-02-13 19:32:22 INFO (SyncWorker_0) [supervisor.docker.interface] Cleaning hassio_cli application
22-02-13 19:32:24 INFO (SyncWorker_0) [supervisor.docker.cli] Starting CLI ghcr.io/home-assistant/aarch64-hassio-cli with version 2021.12.0 - 172.30.32.5
22-02-13 19:32:24 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/aarch64-hassio-dns with version 2021.06.0
22-02-13 19:32:24 INFO (MainThread) [supervisor.plugins.dns] Starting CoreDNS plugin
22-02-13 19:32:24 INFO (SyncWorker_1) [supervisor.docker.interface] Cleaning hassio_dns application
22-02-13 19:32:26 INFO (SyncWorker_1) [supervisor.docker.dns] Starting DNS ghcr.io/home-assistant/aarch64-hassio-dns with version 2021.06.0 - 172.30.32.3
22-02-13 19:32:26 INFO (MainThread) [supervisor.plugins.dns] Updated /etc/resolv.conf
22-02-13 19:32:26 INFO (SyncWorker_0) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/aarch64-hassio-audio with version 2021.07.0
22-02-13 19:32:26 INFO (MainThread) [supervisor.plugins.audio] Starting Audio plugin
22-02-13 19:32:26 INFO (SyncWorker_0) [supervisor.docker.interface] Cleaning hassio_audio application
22-02-13 19:32:27 INFO (SyncWorker_0) [supervisor.docker.audio] Starting Audio ghcr.io/home-assistant/aarch64-hassio-audio with version 2021.07.0 - 172.30.32.4
22-02-13 19:32:27 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/aarch64-hassio-observer with version 2021.10.0
22-02-13 19:32:28 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to ghcr.io/home-assistant/aarch64-hassio-multicast with version 2021.04.0
22-02-13 19:32:28 INFO (MainThread) [supervisor.plugins.multicast] Starting Multicast plugin
22-02-13 19:32:28 INFO (SyncWorker_1) [supervisor.docker.interface] Cleaning hassio_multicast application
22-02-13 19:32:29 INFO (SyncWorker_1) [supervisor.docker.multicast] Starting Multicast ghcr.io/home-assistant/aarch64-hassio-multicast with version 2021.04.0 - Host
dallas6672 commented 2 years ago

Almost 2 weeks on a different microSD card and haven't seen the issue yet.

Could it be possible that there's something with the type, spec or model of microSD that had issues? Or, if that were the case, would it not work at all?

agners commented 2 years ago

Unfortunately SD cards are not that reliable. Could be a bad SD card.

Assuming this was an hardware issue then, closing.