search

home-assistant / operating-system

:beginner: Home Assistant Operating System
Apache License 2.0
5.09k stars 992 forks source link

Core not automatically restarted after hanging #2514

Closed myhades closed 1 year ago

myhades commented 1 year ago

Describe the issue you are experiencing

My core randomly hangs every 1-10 days, and based on the limited information I can found, core container should be automatically restarted in a home assistant os installation? After the core dies, a simple ha core start would do the trick and get it up and running. My setup has gotten way too complex for me to diagnose the problem causing the core to hang, the best i can hope is that those hangs don't have much impact and core resumes quickly. I'm not sure if os is supposed to behave like this and apologize for any inconvenience or confusion. This problem has been noticed around 9.x or so, and all the way to the latest 10.0.

What operating system image do you use?

generic-x86-64 (Generic UEFI capable x86-64 systems)

What version of Home Assistant Operating System is installed?

10.0

Did you upgrade the Operating System.

Yes

Steps to reproduce the issue

Randomly occurs.

Anything in the Supervisor logs that might be useful for us?

2023-04-27 04:47:14.990 WARNING (MainThread) [homeassistant.runner] Task could not be canceled and was still running after shutdown: <Task pending name='Task-1019819' coro=<WebSocketCommonProtocol.close_connection() running at /usr/local/lib/python3.10/site-packages/websockets/legacy/protocol.py:1312> wait_for=<Future pending cb=[shield.<locals>._outer_done_callback() at /usr/local/lib/python3.10/asyncio/tasks.py:864, Task.task_wakeup()]>>
2023-04-27 04:47:15.000 WARNING (MainThread) [homeassistant.runner] Task could not be canceled and was still running after shutdown: <Task pending name='Task-2023632' coro=<RequestHandler.start() running at /usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py:512> wait_for=<Task pending name='Task-2023633' coro=<RequestHandler._handle_request() running at /usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py:433> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[Task.task_wakeup()]>>
2023-04-27 04:47:15.004 WARNING (MainThread) [homeassistant.runner] Task could not be canceled and was still running after shutdown: <Task pending name='Task-2023633' coro=<RequestHandler._handle_request() running at /usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py:433> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[Task.task_wakeup()]>
2023-04-27 04:47:15.114 ERROR (MainThread) [homeassistant] Error doing job: an error occurred during closing of asynchronous generator <async_generator object _acquire at 0x7fcba4feb3c0>
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/runner.py", line 179, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1860, in _run_once
    handle = heapq.heappop(self._scheduled)
RuntimeError: list changed size during iteration

During handling of the above exception, another exception occurred:

RuntimeError: aclose(): asynchronous generator is already running
2023-04-27 04:47:15.117 ERROR (MainThread) [homeassistant] Error doing job: an error occurred during closing of asynchronous generator <async_generator object _acquire at 0x7fcb9fbc4f40>
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/runner.py", line 179, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1860, in _run_once
    handle = heapq.heappop(self._scheduled)
RuntimeError: list changed size during iteration

During handling of the above exception, another exception occurred:

RuntimeError: aclose(): asynchronous generator is already running
2023-04-27 04:47:15.321 WARNING (Thread-10 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_8] is still running at shutdown: File "/usr/local/lib/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 81, in _worker
    work_item = work_queue.get(block=True)
2023-04-27 04:47:15.360 ERROR (MainThread) [homeassistant.helpers.entity] Update for switch.cuco_cp3a_c3c3_switch_4 fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/runner.py", line 179, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1860, in _run_once
    handle = heapq.heappop(self._scheduled)
RuntimeError: list changed size during iteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 550, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 755, in async_device_update
    raise exc
  File "/config/custom_components/xiaomi_miot/switch.py", line 94, in async_update
    await super().async_update()
  File "/config/custom_components/xiaomi_miot/__init__.py", line 1474, in async_update
    await self.async_update_attrs(attrs, update_subs=True)
  File "/config/custom_components/xiaomi_miot/__init__.py", line 1080, in async_update_attrs
    return await self.hass.async_add_executor_job(
  File "/usr/src/homeassistant/homeassistant/core.py", line 555, in async_add_executor_job
    task = self.loop.run_in_executor(None, target, *args)
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 814, in run_in_executor
    self._check_default_executor()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 519, in _check_default_executor
    raise RuntimeError('Executor shutdown has been called')
RuntimeError: Executor shutdown has been called
2023-04-27 04:47:15.493 ERROR (MainThread) [homeassistant.helpers.entity] Update for switch.cuco_cp3a_c3c3_switch_3 fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/runner.py", line 179, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1860, in _run_once
    handle = heapq.heappop(self._scheduled)
RuntimeError: list changed size during iteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 550, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 755, in async_device_update
    raise exc
  File "/config/custom_components/xiaomi_miot/switch.py", line 94, in async_update
    await super().async_update()
  File "/config/custom_components/xiaomi_miot/__init__.py", line 1474, in async_update
    await self.async_update_attrs(attrs, update_subs=True)
  File "/config/custom_components/xiaomi_miot/__init__.py", line 1080, in async_update_attrs
    return await self.hass.async_add_executor_job(
  File "/usr/src/homeassistant/homeassistant/core.py", line 555, in async_add_executor_job
    task = self.loop.run_in_executor(None, target, *args)
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 814, in run_in_executor
    self._check_default_executor()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 519, in _check_default_executor
    raise RuntimeError('Executor shutdown has been called')
RuntimeError: Executor shutdown has been called
2023-04-27 04:47:15.508 ERROR (MainThread) [homeassistant.helpers.entity] Update for switch.cuco_cp3a_c3c3_switch fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/runner.py", line 179, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1860, in _run_once
    handle = heapq.heappop(self._scheduled)
RuntimeError: list changed size during iteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 550, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 755, in async_device_update
    raise exc
  File "/config/custom_components/xiaomi_miot/switch.py", line 94, in async_update
    await super().async_update()
  File "/config/custom_components/xiaomi_miot/__init__.py", line 1478, in async_update
    await self.async_update_attrs({
  File "/config/custom_components/xiaomi_miot/__init__.py", line 1080, in async_update_attrs
    return await self.hass.async_add_executor_job(
  File "/usr/src/homeassistant/homeassistant/core.py", line 555, in async_add_executor_job
    task = self.loop.run_in_executor(None, target, *args)
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 814, in run_in_executor
    self._check_default_executor()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 519, in _check_default_executor
    raise RuntimeError('Executor shutdown has been called')
RuntimeError: Executor shutdown has been called
2023-04-27 04:47:15.730 WARNING (Thread-10 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_14] is still running at shutdown: File "/usr/local/lib/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 83, in _worker
    work_item.run()
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.10/site-packages/miio/click_common.py", line 184, in _wrap
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/miio/integrations/humidifier/deerma/airhumidifier_mjjsq.py", line 157, in status
    values = self.get_properties(properties, max_properties=1)
  File "/usr/local/lib/python3.10/site-packages/miio/device.py", line 240, in get_properties
    values.extend(self.send(property_getter, _props[:max_properties]))
  File "/usr/local/lib/python3.10/site-packages/miio/device.py", line 107, in send
    return self._protocol.send(
  File "/usr/local/lib/python3.10/site-packages/miio/miioprotocol.py", line 193, in send
    data, addr = s.recvfrom(4096)
2023-04-27 04:47:15.784 WARNING (Thread-10 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_11] is still running at shutdown: File "/usr/local/lib/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 83, in _worker
    work_item.run()
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/custom_components/xiaomi_miot/__init__.py", line 655, in get_properties_for_mapping
    return self.get_properties(
  File "/usr/local/lib/python3.10/site-packages/miio/device.py", line 240, in get_properties
    values.extend(self.send(property_getter, _props[:max_properties]))
  File "/usr/local/lib/python3.10/site-packages/miio/device.py", line 107, in send
    return self._protocol.send(
  File "/usr/local/lib/python3.10/site-packages/miio/miioprotocol.py", line 193, in send
    data, addr = s.recvfrom(4096)
2023-04-27 04:47:15.767 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/runner.py", line 179, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1860, in _run_once
    handle = heapq.heappop(self._scheduled)
RuntimeError: list changed size during iteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/storage.py", line 260, in _async_callback_delayed_write
    await self._async_handle_write_data()
  File "/usr/src/homeassistant/homeassistant/helpers/storage.py", line 285, in _async_handle_write_data
    await self._async_write_data(self.path, data)
  File "/usr/src/homeassistant/homeassistant/helpers/storage.py", line 290, in _async_write_data
    await self.hass.async_add_executor_job(self._write_data, self.path, data)
  File "/usr/src/homeassistant/homeassistant/core.py", line 555, in async_add_executor_job
    task = self.loop.run_in_executor(None, target, *args)
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 814, in run_in_executor
    self._check_default_executor()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 519, in _check_default_executor
    raise RuntimeError('Executor shutdown has been called')
RuntimeError: Executor shutdown has been called
2023-04-27 04:47:15.867 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_11] is still running at shutdown: File "/usr/local/lib/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 83, in _worker
    work_item.run()
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/custom_components/xiaomi_miot/__init__.py", line 655, in get_properties_for_mapping
    return self.get_properties(
  File "/usr/local/lib/python3.10/site-packages/miio/device.py", line 240, in get_properties
    values.extend(self.send(property_getter, _props[:max_properties]))
  File "/usr/local/lib/python3.10/site-packages/miio/device.py", line 107, in send
    return self._protocol.send(
  File "/usr/local/lib/python3.10/site-packages/miio/miioprotocol.py", line 193, in send
    data, addr = s.recvfrom(4096)
2023-04-27 04:47:23.301 ERROR (DHCPWatcher) [root] Uncaught thread exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.10/site-packages/scapy/sendrecv.py", line 1254, in _run
    session.on_packet_received(p)
  File "/usr/local/lib/python3.10/site-packages/scapy/sessions.py", line 109, in on_packet_received
    result = self.prn(pkt)
  File "/usr/src/homeassistant/homeassistant/components/dhcp/__init__.py", line 450, in _handle_dhcp_packet
    self.process_client(ip_address, hostname, mac_address)
  File "/usr/src/homeassistant/homeassistant/components/dhcp/__init__.py", line 144, in process_client
    return run_callback_threadsafe(
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 46, in run_callback_threadsafe
    loop.call_soon_threadsafe(run_callback)
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 798, in call_soon_threadsafe
    self._check_closed()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

Anything in the Host logs that might be useful for us?

Apr 26 15:55:00 myhades-ha systemd[1]: Started Network Manager Script Dispatcher Service.
Apr 26 15:55:00 myhades-ha NetworkManager[363]: <info>  [1682524500.9810] manager: NetworkManager state is now CONNECTED_GLOBAL
Apr 26 15:55:11 myhades-ha systemd[1]: NetworkManager-dispatcher.service: Deactivated successfully.
Apr 26 17:24:10 myhades-ha kernel: audit: type=1334 audit(1682529850.589:2010): prog-id=462 op=LOAD
Apr 26 17:24:10 myhades-ha kernel: audit: type=1334 audit(1682529850.590:2011): prog-id=463 op=LOAD
Apr 26 17:24:10 myhades-ha kernel: audit: type=1334 audit(1682529850.590:2012): prog-id=464 op=LOAD
Apr 26 17:24:10 myhades-ha systemd[1]: Starting Hostname Service...
Apr 26 17:24:11 myhades-ha systemd[1]: Started Hostname Service.
Apr 26 17:24:11 myhades-ha kernel: audit: type=1334 audit(1682529851.494:2013): prog-id=465 op=LOAD
Apr 26 17:24:11 myhades-ha kernel: audit: type=1334 audit(1682529851.494:2014): prog-id=466 op=LOAD
Apr 26 17:24:11 myhades-ha kernel: audit: type=1334 audit(1682529851.495:2015): prog-id=467 op=LOAD
Apr 26 17:24:11 myhades-ha systemd[1]: Starting Time & Date Service...
Apr 26 17:24:12 myhades-ha systemd[1]: Started Time & Date Service.
Apr 26 17:24:41 myhades-ha systemd[1]: systemd-hostnamed.service: Deactivated successfully.
Apr 26 17:24:41 myhades-ha kernel: audit: type=1334 audit(1682529881.516:2016): prog-id=464 op=UNLOAD
Apr 26 17:24:41 myhades-ha kernel: audit: type=1334 audit(1682529881.516:2017): prog-id=463 op=UNLOAD
Apr 26 17:24:41 myhades-ha kernel: audit: type=1334 audit(1682529881.516:2018): prog-id=462 op=UNLOAD
Apr 26 17:24:42 myhades-ha systemd[1]: systemd-timedated.service: Deactivated successfully.
Apr 26 17:24:42 myhades-ha kernel: audit: type=1334 audit(1682529882.294:2019): prog-id=467 op=UNLOAD
Apr 26 17:24:42 myhades-ha kernel: audit: type=1334 audit(1682529882.294:2020): prog-id=466 op=UNLOAD
Apr 26 17:24:42 myhades-ha kernel: audit: type=1334 audit(1682529882.294:2021): prog-id=465 op=UNLOAD
Apr 26 19:30:52 myhades-ha kernel: audit: type=1334 audit(1682537452.847:2022): prog-id=468 op=LOAD
Apr 26 19:30:52 myhades-ha kernel: audit: type=1334 audit(1682537452.847:2023): prog-id=469 op=LOAD
Apr 26 19:30:52 myhades-ha kernel: audit: type=1334 audit(1682537452.848:2024): prog-id=470 op=LOAD
Apr 26 19:30:52 myhades-ha systemd[1]: Starting Hostname Service...
Apr 26 19:30:53 myhades-ha systemd[1]: Started Hostname Service.
Apr 26 19:30:54 myhades-ha kernel: audit: type=1334 audit(1682537454.049:2025): prog-id=471 op=LOAD
Apr 26 19:30:54 myhades-ha kernel: audit: type=1334 audit(1682537454.050:2026): prog-id=472 op=LOAD
Apr 26 19:30:54 myhades-ha kernel: audit: type=1334 audit(1682537454.050:2027): prog-id=473 op=LOAD
Apr 26 19:30:54 myhades-ha systemd[1]: Starting Time & Date Service...
Apr 26 19:30:55 myhades-ha systemd[1]: Started Time & Date Service.
Apr 26 19:31:23 myhades-ha systemd[1]: systemd-hostnamed.service: Deactivated successfully.
Apr 26 19:31:23 myhades-ha kernel: audit: type=1334 audit(1682537483.936:2028): prog-id=470 op=UNLOAD
Apr 26 19:31:23 myhades-ha kernel: audit: type=1334 audit(1682537483.936:2029): prog-id=469 op=UNLOAD
Apr 26 19:31:23 myhades-ha kernel: audit: type=1334 audit(1682537483.936:2030): prog-id=468 op=UNLOAD
Apr 26 19:31:25 myhades-ha systemd[1]: systemd-timedated.service: Deactivated successfully.
Apr 26 19:31:25 myhades-ha kernel: audit: type=1334 audit(1682537485.107:2031): prog-id=471 op=UNLOAD
Apr 26 19:31:25 myhades-ha kernel: audit: type=1334 audit(1682537485.112:2032): prog-id=473 op=UNLOAD
Apr 26 19:31:25 myhades-ha kernel: audit: type=1334 audit(1682537485.112:2033): prog-id=472 op=UNLOAD
Apr 26 19:47:40 myhades-ha NetworkManager[363]: <info>  [1682538460.6846] dhcp4 (enp0s3): state changed new lease, address=192.168.50.200
Apr 26 20:47:42 myhades-ha systemd[1]: docker-4c994f328247d32353ff41370bb06ec99fc3bc43ba7c5c78fcd7300d051a64d0.scope: Deactivated successfully.
Apr 26 20:47:42 myhades-ha systemd[1]: docker-4c994f328247d32353ff41370bb06ec99fc3bc43ba7c5c78fcd7300d051a64d0.scope: Consumed 6h 53min 55.078s CPU time.
Apr 26 20:47:42 myhades-ha dockerd[456]: time="2023-04-26T20:47:42.846785010Z" level=info msg="ignoring event" container=4c994f328247d32353ff41370bb06ec99fc3bc43ba7c5c78fcd7300d051a64d0 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Apr 26 20:47:43 myhades-ha systemd[1]: var-lib-docker-overlay2-ba7248b969b9e0e949a50af518284e7e068d82d9f36394a60e6b6505fdcbe53b-merged.mount: Deactivated successfully.
Apr 26 20:47:43 myhades-ha systemd[1]: mnt-data-docker-overlay2-ba7248b969b9e0e949a50af518284e7e068d82d9f36394a60e6b6505fdcbe53b-merged.mount: Deactivated successfully.
Apr 26 20:49:39 myhades-ha kernel: audit: type=1334 audit(1682542179.786:2034): prog-id=325 op=UNLOAD
Apr 26 21:37:35 myhades-ha kernel: audit: type=1334 audit(1682545055.887:2035): prog-id=474 op=LOAD
Apr 26 21:37:35 myhades-ha kernel: audit: type=1334 audit(1682545055.888:2036): prog-id=475 op=LOAD
Apr 26 21:37:35 myhades-ha kernel: audit: type=1334 audit(1682545055.888:2037): prog-id=476 op=LOAD
Apr 26 21:37:35 myhades-ha systemd[1]: Starting Hostname Service...
Apr 26 21:37:37 myhades-ha systemd[1]: Started Hostname Service.
Apr 26 21:37:37 myhades-ha kernel: audit: type=1334 audit(1682545057.351:2038): prog-id=477 op=LOAD
Apr 26 21:37:37 myhades-ha kernel: audit: type=1334 audit(1682545057.353:2039): prog-id=478 op=LOAD
Apr 26 21:37:37 myhades-ha kernel: audit: type=1334 audit(1682545057.353:2040): prog-id=479 op=LOAD
Apr 26 21:37:37 myhades-ha systemd[1]: Starting Time & Date Service...
Apr 26 21:37:38 myhades-ha systemd[1]: Started Time & Date Service.
Apr 26 21:38:07 myhades-ha systemd[1]: systemd-hostnamed.service: Deactivated successfully.
Apr 26 21:38:07 myhades-ha kernel: audit: type=1334 audit(1682545087.406:2041): prog-id=476 op=UNLOAD
Apr 26 21:38:07 myhades-ha kernel: audit: type=1334 audit(1682545087.407:2042): prog-id=475 op=UNLOAD
Apr 26 21:38:07 myhades-ha kernel: audit: type=1334 audit(1682545087.408:2043): prog-id=474 op=UNLOAD
Apr 26 21:38:08 myhades-ha systemd[1]: systemd-timedated.service: Deactivated successfully.
Apr 26 21:38:08 myhades-ha kernel: audit: type=1334 audit(1682545088.157:2044): prog-id=479 op=UNLOAD
Apr 26 21:38:08 myhades-ha kernel: audit: type=1334 audit(1682545088.157:2045): prog-id=478 op=UNLOAD
Apr 26 21:38:08 myhades-ha kernel: audit: type=1334 audit(1682545088.157:2046): prog-id=477 op=UNLOAD
Apr 26 23:44:19 myhades-ha kernel: audit: type=1334 audit(1682552659.106:2047): prog-id=480 op=LOAD
Apr 26 23:44:19 myhades-ha kernel: audit: type=1334 audit(1682552659.106:2048): prog-id=481 op=LOAD
Apr 26 23:44:19 myhades-ha kernel: audit: type=1334 audit(1682552659.107:2049): prog-id=482 op=LOAD
Apr 26 23:44:19 myhades-ha systemd[1]: Starting Hostname Service...
Apr 26 23:44:19 myhades-ha systemd[1]: Started Hostname Service.
Apr 26 23:44:19 myhades-ha kernel: audit: type=1334 audit(1682552659.979:2050): prog-id=483 op=LOAD
Apr 26 23:44:19 myhades-ha kernel: audit: type=1334 audit(1682552659.979:2051): prog-id=484 op=LOAD
Apr 26 23:44:19 myhades-ha kernel: audit: type=1334 audit(1682552659.980:2052): prog-id=485 op=LOAD
Apr 26 23:44:20 myhades-ha systemd[1]: Starting Time & Date Service...
Apr 26 23:44:20 myhades-ha systemd[1]: Started Time & Date Service.
Apr 26 23:44:49 myhades-ha systemd[1]: systemd-hostnamed.service: Deactivated successfully.
Apr 26 23:44:49 myhades-ha kernel: audit: type=1334 audit(1682552689.947:2053): prog-id=482 op=UNLOAD
Apr 26 23:44:49 myhades-ha kernel: audit: type=1334 audit(1682552689.947:2054): prog-id=481 op=UNLOAD
Apr 26 23:44:49 myhades-ha kernel: audit: type=1334 audit(1682552689.947:2055): prog-id=480 op=UNLOAD
Apr 26 23:44:50 myhades-ha systemd[1]: systemd-timedated.service: Deactivated successfully.
Apr 26 23:44:50 myhades-ha kernel: audit: type=1334 audit(1682552690.690:2056): prog-id=485 op=UNLOAD
Apr 26 23:44:50 myhades-ha kernel: audit: type=1334 audit(1682552690.690:2057): prog-id=484 op=UNLOAD
Apr 26 23:44:50 myhades-ha kernel: audit: type=1334 audit(1682552690.690:2058): prog-id=483 op=UNLOAD
Apr 27 00:17:28 myhades-ha systemd[1]: run-docker-runtime\x2drunc-moby-4c994f328247d32353ff41370bb06ec99fc3bc43ba7c5c78fcd7300d051a64d0-runc.6NtOvh.mount: Deactivated successfully.
Apr 27 00:17:28 myhades-ha systemd[1]: Started libcontainer container 4c994f328247d32353ff41370bb06ec99fc3bc43ba7c5c78fcd7300d051a64d0.
Apr 27 00:17:28 myhades-ha kernel: audit: type=1334 audit(1682554648.716:2059): prog-id=486 op=LOAD
Apr 27 00:17:28 myhades-ha kernel: audit: type=1300 audit(1682554648.716:2059): arch=c000003e syscall=321 success=yes exit=15 a0=5 a1=c00018d7f8 a2=78 a3=0 items=0 ppid=343168 pid=343177 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runc" exe="/usr/bin/runc" subj=unconfined key=(null)
Apr 27 00:17:28 myhades-ha kernel: audit: type=1327 audit(1682554648.716:2059): proctitle=72756E63002D2D726F6F74002F7661722F72756E2F646F636B65722F72756E74696D652D72756E632F6D6F6279002D2D6C6F67002F72756E2F636F6E7461696E6572642F696F2E636F6E7461696E6572642E72756E74696D652E76322E7461736B2F6D6F62792F34633939346633323832343764333233353366663431333730
Apr 27 00:17:28 myhades-ha kernel: audit: type=1334 audit(1682554648.717:2060): prog-id=487 op=LOAD
Apr 27 00:17:28 myhades-ha kernel: audit: type=1300 audit(1682554648.717:2060): arch=c000003e syscall=321 success=yes exit=17 a0=5 a1=c00018d590 a2=78 a3=0 items=0 ppid=343168 pid=343177 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runc" exe="/usr/bin/runc" subj=unconfined key=(null)
Apr 27 00:17:28 myhades-ha kernel: audit: type=1327 audit(1682554648.717:2060): proctitle=72756E63002D2D726F6F74002F7661722F72756E2F646F636B65722F72756E74696D652D72756E632F6D6F6279002D2D6C6F67002F72756E2F636F6E7461696E6572642F696F2E636F6E7461696E6572642E72756E74696D652E76322E7461736B2F6D6F62792F34633939346633323832343764333233353366663431333730
Apr 27 00:17:28 myhades-ha kernel: audit: type=1334 audit(1682554648.719:2061): prog-id=487 op=UNLOAD
Apr 27 00:17:28 myhades-ha kernel: audit: type=1334 audit(1682554648.719:2062): prog-id=486 op=UNLOAD
Apr 27 00:17:28 myhades-ha kernel: audit: type=1334 audit(1682554648.719:2063): prog-id=488 op=LOAD
Apr 27 00:17:28 myhades-ha kernel: audit: type=1300 audit(1682554648.719:2063): arch=c000003e syscall=321 success=yes exit=15 a0=5 a1=c00018da50 a2=78 a3=0 items=0 ppid=343168 pid=343177 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runc" exe="/usr/bin/runc" subj=unconfined key=(null)
Apr 27 00:20:57 myhades-ha bluetoothd[364]: src/device.c:device_add_connection() Device F8:24:41:E8:03:3D is already connected
Apr 27 01:25:25 myhades-ha kernel: kauditd_printk_skb: 1 callbacks suppressed
Apr 27 01:25:25 myhades-ha kernel: audit: type=1334 audit(1682558725.163:2064): prog-id=489 op=LOAD
Apr 27 01:25:25 myhades-ha systemd[1]: Started Journal Gateway Service.
Apr 27 01:25:25 myhades-ha systemd-timesyncd[169704]: Network configuration changed, trying to establish connection.
Apr 27 01:25:25 myhades-ha systemd-timesyncd[169704]: Contacted time server 162.159.200.123:123 (time.cloudflare.com).
Apr 27 01:25:26 myhades-ha systemd-journal-gatewayd[347796]: microhttpd: MHD_OPTION_EXTERNAL_LOGGER is not the first option specified for the daemon. Some messages may be printed by the standard MHD logger.

System information

System Information

version core-2023.4.6
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.10.10
os_name Linux
os_version 6.1.24
arch x86_64
timezone Asia/Shanghai
config_dir /config
Home Assistant Community Store GitHub API | ok -- | -- GitHub Content | ok GitHub Web | ok GitHub API Calls Remaining | 5000 Installed Version | 1.32.1 Stage | running Available Repositories | 1337 Downloaded Repositories | 49
Home Assistant Supervisor host_os | Home Assistant OS 10.0 -- | -- update_channel | stable supervisor_version | supervisor-2023.04.1 agent_version | 1.5.1 docker_version | 23.0.3 disk_total | 30.8 GB disk_used | 19.2 GB healthy | true supported | true board | ova supervisor_api | ok version_api | ok installed_addons | AirCast (3.5.2), Studio Code Server (5.5.3), Samba share (10.0.1), Terminal & SSH (9.6.1), File editor (5.5.0), Node-RED (14.1.0), Rclone Backup (3.0.1), chrony (2.5.0), VNC Viewer (1.6.0), ESPHome (2023.3.0), Mosquitto broker (6.2.0)
Dashboards dashboards | 7 -- | -- resources | 19 views | 50 mode | storage
Recorder oldest_recorder_run | April 17, 2023 at 2:08 PM -- | -- current_recorder_run | April 27, 2023 at 8:18 AM estimated_db_size | 1092.58 MiB database_engine | sqlite database_version | 3.38.5
Spotify api_endpoint_reachable | ok -- | --
Xiaomi Miot Auto component_version | 0.7.7 -- | -- can_reach_server | ok can_reach_spec | ok logged_accounts | 0 total_devices | 0
Xiaomi MIoT logged_in | false -- | -- added_devices | 2

Additional information

IMG_0373 ha core restart runs smoothly. One thing to notice, host log time is 8 hours behind the supervisor, guess it's using utc time. the hang occur roughly 8pm 4.26 utc and 4pm 4.27 at my location.

github-actions[bot] commented 1 year ago

:wave: @myhades, thanks for reporting an issue!

It looks like this issue is related to Home Assistant Supervisor. Please check the Home Assistant Supervisor repository, the issue might have been reported already. Open a new issue in that repository if you can't find a matching issue.

agners commented 1 year ago

Not automatically restarting is something which should be reported to the Supervisor repository. That said: Your ATA errors look somewhat concerning. This might be a hardware defect.

myhades commented 1 year ago

Not automatically restarting is something which should be reported to the Supervisor repository. That said: Your ATA errors look somewhat concerning. This might be a hardware defect.

Thanks for the advice, will move the issue there. As for the ata error, no idea what does that mean in a virtual machine with a virtual disk (host is perfectly fine). I always got it backed up though if there are problems. But thanks again. ;)