home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
74.19k stars 31.15k forks source link

Home assistant has crashed! #114849

Closed mrrouge closed 5 months ago

mrrouge commented 8 months ago

The problem

Suddently HA decided to crash, no more GUI. I still have access to SSH. Already tried to reboot NUC and Router, core rebuild, and supervisor repair. Any help is very appreciated!! Thanks!

What version of Home Assistant Core has the issue?

12.1

What was the last working version of Home Assistant Core?

2024.3.3

What type of installation are you running?

Home Assistant OS

Integration causing the issue

No idea

Link to integration documentation on our website

No response

Diagnostics information

`~ ha supervisor logs ... 2024-04-04 00:25:08.859 INFO (MainThread) [supervisor.homeassistant.core] Home Assistant Core state changed to NOT_RUNNING 2024-04-04 00:25:08.860 INFO (MainThread) [supervisor.auth] Home Assistant not running, checking cache 2024-04-04 00:25:49.313 ERROR (MainThread) [supervisor.homeassistant.api] Error on call http://172.30.32.1:8123/api/core/state: [Errno 104] Connection reset by peer 2024-04-04 00:25:49.319 ERROR (MainThread) [supervisor.homeassistant.api] Error on call http://172.30.32.1:8123/api/core/state: [Errno 104] Connection reset by peer 2024-04-04 00:25:49.332 ERROR (MainThread) [supervisor.homeassistant.api]Error on call http://172.30.32.1:8123/api/core/state: [Errno 104] Connection reset by peer 2024-04-04 00:25:54.320 ERROR (MainThread) [supervisor.homeassistant.core] Home Assistant has crashed

~ ha core logs 2024-04-04 00:25:03.557 ERROR (MainThread) [homeassistant.helpers.translation] Validation of translation placeholders for localized (it) string component.rest_command.exceptions.client_error.message failed 2024-04-04 00:25:03.557 ERROR (MainThread) [homeassistant.helpers.translation] Validation of translation placeholders for localized (it) string component.rest_command.exceptions.decoding_error.message failed 2024-04-04 00:25:03.557 ERROR (MainThread) [homeassistant.helpers.translation] Validation of translation placeholders for localized (it) string component.rest_command.exceptions.timeout.message failed 2024-04-04 00:25:04.190 WARNING (Recorder) [homeassistant.components.recorder.util] The system could not validate that the sqlite3 database at //config/home-assistant_v2.db was shutdown cleanly 2024-04-04 00:25:04.211 WARNING (Recorder) [homeassistant.components.recorder.util] Ended unfinished session (id=269 from 2024-04-03 13:19:50.260272) 2024-04-04 00:25:07.290 INFO (SyncWorker_0) [fritzconnection] Unable to retrieve resource 'http://192.168.1.1:49000/tr64desc.xml' from the device. 2024-04-04 00:25:08.844 WARNING (MainThread) [homeassistant.helpers.frame] Detected that custom integration 'browser_mod' accesses hass.components.frontend. This is deprecated and will stop working in Home Assistant 2024.9, it should be updated to import functions used from frontend directly at custom_components/browser_mod/mod_view.py, line 24: hass.components.frontend.async_register_built_in_panel(, please report it to the author of the 'browser_mod' custom integration 2024-04-04 00:25:09.636 WARNING (MainThread) [homeassistant.components.light] Entity None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <LightEntityFeature: 1> and color modes, please create a bug report at https://github.com/rroller/dahua/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation 2024-04-04 00:25:09.636 WARNING (MainThread) [homeassistant.components.light] None (<class 'custom_components.dahua.light.DahuaInfraredLight'>) does not set supported color modes, this will stop working in Home Assistant Core 2025.3, please create a bug report at https://github.com/rroller/dahua/issues [22:25:49] INFO: Home Assistant Core finish process exit code 256 [22:25:49] INFO: Home Assistant Core finish process received signal 9 s6-rc: info: service legacy-services: stopping s6-rc: info: service legacy-services successfully stopped s6-rc: info: service legacy-cont-init: stopping s6-rc: info: service legacy-cont-init successfully stopped s6-rc: info: service fix-attrs: stopping s6-rc: info: service fix-attrs successfully stopped s6-rc: info: service s6rc-oneshot-runner: stopping s6-rc: info: service s6rc-oneshot-runner successfully stopped `

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

~ ha info arch: amd64 channel: stable docker: 24.0.7 features:

smoki3 commented 8 months ago

Have the same issue. Since 2 days it crashes often. Only a reboot fix the problem

mrrouge commented 8 months ago

Unfortunately not even rebooting works for me..

collse commented 8 months ago

I have similar issues and have rolled back - cant seem to identify where the crashes come from in my logs nothing is written prior that would be consistent and identifies the issue

mrrouge commented 8 months ago

I just found only partial backups, and it seems I cannot restore any of them in SSH..

mrrouge commented 8 months ago

Solved by restoring partial backups

joostlek commented 8 months ago

There should be logs on the file system, we need them to identify the issue

mrrouge commented 8 months ago

Spoke too soon. Restored partial backup of zigbee2mqtt upgrade, system come up, GUI was loading before it lost connection again.

mrrouge commented 8 months ago

There should be logs on the file system, we need them to identify the issue

Please can you help me share the right logs?

joostlek commented 8 months ago

There should be logs that end with .fault probably.

Otherwise please ask around on discord, I'm not available to help with this in the following day

mrrouge commented 8 months ago

Yepp, don't worry. .fault is empty, but i found this in home-assistant.log: 2024-04-04 23:46:20.045 ERROR (MainThread) [frontend.js.latest.202404031] Uncaught error from Firefox 124.0 on Windows 10 Error: Failed to execute 'define' on 'CustomElementRegistry': the name "action-handler-clock-weather" has already been used with this registry 3900/</window.CustomElementRegistry.prototype.define (src/scoped-custom-element-registry.js:44:14) /hacsfiles/clock-weather-card/clock-weather-card.js:175:2352

collse commented 8 months ago

my .fault is also empty, my logger setup is as follows:

logger: default: info logs: sucks: debug homeassistant.components.automation: debug homeassistant.core: debug homeassistant.supervisor: debug pyaarlo: debug custom_components.aarlo: debug custom_components.aarlo.alarm_control_panel: debug custom_components.aarlo.binary_sensor: debug custom_components.aarlo.camera: debug custom_components.aarlo.light: debug custom_components.aarlo.media_player: debug custom_components.aarlo.sensor: debug custom_components.aarlo.switch: debug aioftp.client: notset

and nothing is being captured of relevance that I could pinpoint to a particular issue, initial warnings are the common ones: 2024-04-04 13:45:35.309 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration watchman which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.310 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration waste_collection_schedule which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.310 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration bestway which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.311 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration xiaomi_cloud_map_extractor which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.311 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration webrtc which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.312 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration scheduler which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.312 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration aarlo which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.313 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration spook_inverse which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.313 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration spook which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.314 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration yi_hack which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.314 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration deebot which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.315 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration alexa_media which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.315 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration wiser which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.315 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration govee_lan which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.316 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration octopus_energy which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.316 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration scrypted which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant 2024-04-04 13:45:35.317 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant

mrrouge commented 8 months ago

try this:

touch /config/safe-mode
ha core restart
# HA boots up in safe mode, everything orange, integrations not loaded
ha core restart
# HA boots up as normal

It worked for me

collse commented 8 months ago

try this:

touch /config/safe-mode
ha core restart
# HA boots up in safe mode, everything orange, integrations not loaded
ha core restart
# HA boots up as normal

It worked for me

checking if it does, I will run it for some time in safe mode to see if it still crashes and if it doesnt I will run it in live mode

collse commented 8 months ago

it did not - just died again ...

in the logs I can see entries which may be relevant but I am not sure ...

2024-04-05 00:32:34.360 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f391c0081f0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name="event trigger {'domain': 'automation', 'name': 'Vacuum - Update Map Extractor camera', 'home_assistant_start': True, 'variables': {'this': {'entity_id': 'automation.vacuum_update_map_extractor_camera', 'state': 'on', 'attributes': {'id': '1708260130868', 'last_triggered': datetime.datetime(2024, 4, 4, 22, 7, 52, 138404, tzinfo=datetime.timezone.utc), 'mode': 'parallel', 'current': 0, 'max': 10, 'friendly_name': 'Vacuum - Update Map Extractor camera', 'icon': 'mdi:arrow-right-drop-circle'}, 'last_changed': '2024-04-04T23:28:42.638574+00:00', 'last_reported': '2024-04-04T23:28:42.638737+00:00', 'last_updated': '2024-04-04T23:28:42.638574+00:00', 'context': {'id': '01HTNQ2WME95YPFVKKNZR4TC15', 'parent_id': None, 'user_id': None}}}, 'trigger_data': {'id': 'state_changed', 'idx': '0', 'alias': None}}" coro=<AutomationEntity.async_trigger() running at /usr/src/homeassistant/homeassistant/components/automation/init.py:673>> is being executed. 2024-04-05 00:32:34.362 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-3120' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> while another task <Task pending name="event trigger {'domain': 'automation', 'name': 'Vacuum - Update Map Extractor camera', 'home_assistant_start': True, 'variables': {'this': {'entity_id': 'automation.vacuum_update_map_extractor_camera', 'state': 'on', 'attributes': {'id': '1708260130868', 'last_triggered': datetime.datetime(2024, 4, 4, 22, 7, 52, 138404, tzinfo=datetime.timezone.utc), 'mode': 'parallel', 'current': 0, 'max': 10, 'friendly_name': 'Vacuum - Update Map Extractor camera', 'icon': 'mdi:arrow-right-drop-circle'}, 'last_changed': '2024-04-04T23:28:42.638574+00:00', 'last_reported': '2024-04-04T23:28:42.638737+00:00', 'last_updated': '2024-04-04T23:28:42.638574+00:00', 'context': {'id': '01HTNQ2WME95YPFVKKNZR4TC15', 'parent_id': None, 'user_id': None}}}, 'trigger_data': {'id': 'state_changed', 'idx': '0', 'alias': None}}" coro=<AutomationEntity.async_trigger() running at /usr/src/homeassistant/homeassistant/components/automation/init.py:673>> is being executed. 2024-04-05 00:32:34.363 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f391c00a200>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name="event trigger {'domain': 'automation', 'name': 'Vacuum - Update Map Extractor camera', 'home_assistant_start': True, 'variables': {'this': {'entity_id': 'automation.vacuum_update_map_extractor_camera', 'state': 'on', 'attributes': {'id': '1708260130868', 'last_triggered': datetime.datetime(2024, 4, 4, 22, 7, 52, 138404, tzinfo=datetime.timezone.utc), 'mode': 'parallel', 'current': 0, 'max': 10, 'friendly_name': 'Vacuum - Update Map Extractor camera', 'icon': 'mdi:arrow-right-drop-circle'}, 'last_changed': '2024-04-04T23:28:42.638574+00:00', 'last_reported': '2024-04-04T23:28:42.638737+00:00', 'last_updated': '2024-04-04T23:28:42.638574+00:00', 'context': {'id': '01HTNQ2WME95YPFVKKNZR4TC15', 'parent_id': None, 'user_id': None}}}, 'trigger_data': {'id': 'state_changed', 'idx': '0', 'alias': None}}" coro=<AutomationEntity.async_trigger() running at /usr/src/homeassistant/homeassistant/components/automation/init.py:673>> is being executed. 2024-04-05 00:32:34.364 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f391c0521d0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name="event trigger {'domain': 'automation', 'name': 'Vacuum - Update Map Extractor camera', 'home_assistant_start': True, 'variables': {'this': {'entity_id': 'automation.vacuum_update_map_extractor_camera', 'state': 'on', 'attributes': {'id': '1708260130868', 'last_triggered': datetime.datetime(2024, 4, 4, 22, 7, 52, 138404, tzinfo=datetime.timezone.utc), 'mode': 'parallel', 'current': 0, 'max': 10, 'friendly_name': 'Vacuum - Update Map Extractor camera', 'icon': 'mdi:arrow-right-drop-circle'}, 'last_changed': '2024-04-04T23:28:42.638574+00:00', 'last_reported': '2024-04-04T23:28:42.638737+00:00', 'last_updated': '2024-04-04T23:28:42.638574+00:00', 'context': {'id': '01HTNQ2WME95YPFVKKNZR4TC15', 'parent_id': None, 'user_id': None}}}, 'trigger_data': {'id': 'state_changed', 'idx': '0', 'alias': None}}" coro=<AutomationEntity.async_trigger() running at /usr/src/homeassistant/homeassistant/components/automation/init.py:673>> is being executed. 2024-04-05 00:32:34.371 DEBUG (ArloBackgroundWorker) [homeassistant.components.automation.vacuum_update_map_extractor_camera] Conditions not met, aborting automation. Condition summary: {'trigger/0': deque([{'path': 'trigger/0', 'timestamp': datetime.datetime(2024, 4, 4, 23, 32, 34, 370149, tzinfo=datetime.timezone.utc), 'changed_variables': {'this': {'entity_id': 'automation.vacuum_update_map_extractor_camera', 'state': 'on', 'attributes': {'id': '1708260130868', 'last_triggered': datetime.datetime(2024, 4, 4, 23, 30, 44, 151454, tzinfo=datetime.timezone.utc), 'mode': 'parallel', 'current': 0, 'max': 10, 'friendly_name': 'Vacuum - Update Map Extractor camera', 'icon': 'mdi:arrow-right-drop-circle'}, 'last_changed': '2024-04-04T23:28:42.638574+00:00', 'last_reported': '2024-04-04T23:30:50.534010+00:00', 'last_updated': '2024-04-04T23:30:50.534010+00:00', 'context': {'id': '01HTNQ6SH6CY9RRC32KHVGV6AS', 'parent_id': None, 'user_id': None}}, 'trigger': {'id': 'state_changed', 'idx': '0', 'alias': None, 'platform': 'event', 'event': <Event state_changed[L]: entity_id=binary_sensor.aarlo_motion_hallway, old_state=<state binary_sensor.aarlo_motion_hallway=off; attribution=Data provided by my.arlo.com, name=Motion Hallway, device_brand=Arlo, device_name=Hallway, device_id=59U17B7RA91A6, device_model=VMC4030, device_class=motion, friendly_name=Motion Hallway @ 2024-04-05T00:28:40.482333+01:00>, new_state=<state binary_sensor.aarlo_motion_hallway=on; attribution=Data provided by my.arlo.com, name=Motion Hallway, device_brand=Arlo, device_name=Hallway, device_id=59U17B7RA91A6, device_model=VMC4030, device_class=motion, friendly_name=Motion Hallway @ 2024-04-05T00:32:34.350178+01:00>>, 'description': "event 'state_changed'"}, 'interval_docked': 1800, 'interval_cleaning': 10, 'mode_to_handle': 'browser_mod_path', 'update_once_after_docking': True, 'vacuum_entity': 'vacuum.robot_vacuum_s5', 'camera_entity': 'camera.xiaomi_cloud_map_extractor', 'browser_mod_path': '/lovelace/vacuum', 'is_vacuum_docked': True, 'is_interval_docked_trigger': False, 'is_interval_cleaning_trigger': False, 'is_vacuum_state_trigger': False, 'is_browser_mod_template_trigger': False, 'is_vacuum_docked_trigger': False, 'active_browsers': 0}}]), 'condition/0': deque([{'path': 'condition/0', 'timestamp': datetime.datetime(2024, 4, 4, 23, 32, 34, 370243, tzinfo=datetime.timezone.utc), 'result': {'result': False}}]), 'condition/0/conditions/0': deque([{'path': 'condition/0/conditions/0', 'timestamp': datetime.datetime(2024, 4, 4, 23, 32, 34, 370302, tzinfo=datetime.timezone.utc), 'result': {'result': False, 'entities': []}}]), 'condition/0/conditions/1': deque([{'path': 'condition/0/conditions/1', 'timestamp': datetime.datetime(2024, 4, 4, 23, 32, 34, 370575, tzinfo=datetime.timezone.utc), 'result': {'result': False, 'entities': []}}]), 'condition/0/conditions/2': deque([{'path': 'condition/0/conditions/2', 'timestamp': datetime.datetime(2024, 4, 4, 23, 32, 34, 370761, tzinfo=datetime.timezone.utc), 'result': {'result': False, 'entities': []}}]), 'condition/0/conditions/3': deque([{'path': 'condition/0/conditions/3', 'timestamp': datetime.datetime(2024, 4, 4, 23, 32, 34, 370923, tzinfo=datetime.timezone.utc), 'result': {'result': False, 'entities': []}}]), 'condition/0/conditions/4': deque([{'path': 'condition/0/conditions/4', 'timestamp': datetime.datetime(2024, 4, 4, 23, 32, 34, 371080, tzinfo=datetime.timezone.utc), 'result': {'result': False, 'entities': []}}])}

as well as:

2024-04-05 00:36:41.941 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f391c001360>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name="event trigger {'domain': 'automation', 'name': 'Vacuum - Update Map Extractor camera', 'home_assistant_start': True, 'variables': {'this': {'entity_id': 'automation.vacuum_update_map_extractor_camera', 'state': 'on', 'attributes': {'id': '1708260130868', 'last_triggered': datetime.datetime(2024, 4, 4, 22, 7, 52, 138404, tzinfo=datetime.timezone.utc), 'mode': 'parallel', 'current': 0, 'max': 10, 'friendly_name': 'Vacuum - Update Map Extractor camera', 'icon': 'mdi:arrow-right-drop-circle'}, 'last_changed': '2024-04-04T23:28:42.638574+00:00', 'last_reported': '2024-04-04T23:28:42.638737+00:00', 'last_updated': '2024-04-04T23:28:42.638574+00:00', 'context': {'id': '01HTNQ2WME95YPFVKKNZR4TC15', 'parent_id': None, 'user_id': None}}}, 'trigger_data': {'id': 'state_changed', 'idx': '0', 'alias': None}}" coro=<AutomationEntity.async_trigger() running at /usr/src/homeassistant/homeassistant/components/automation/init.py:673>> is being executed. 2024-04-05 00:36:41.948 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='EntityPlatform poll media_player.androidtv' coro=<EntityPlatform._update_entity_states() running at /usr/src/homeassistant/homeassistant/helpers/entity_platform.py:1018> wait_for= cb=[set.remove(), set.remove()]> while another task <Task pending name="event trigger {'domain': 'automation', 'name': 'Vacuum - Update Map Extractor camera', 'home_assistant_start': True, 'variables': {'this': {'entity_id': 'automation.vacuum_update_map_extractor_camera', 'state': 'on', 'attributes': {'id': '1708260130868', 'last_triggered': datetime.datetime(2024, 4, 4, 22, 7, 52, 138404, tzinfo=datetime.timezone.utc), 'mode': 'parallel', 'current': 0, 'max': 10, 'friendly_name': 'Vacuum - Update Map Extractor camera', 'icon': 'mdi:arrow-right-drop-circle'}, 'last_changed': '2024-04-04T23:28:42.638574+00:00', 'last_reported': '2024-04-04T23:28:42.638737+00:00', 'last_updated': '2024-04-04T23:28:42.638574+00:00', 'context': {'id': '01HTNQ2WME95YPFVKKNZR4TC15', 'parent_id': None, 'user_id': None}}}, 'trigger_data': {'id': 'state_changed', 'idx': '0', 'alias': None}}" coro=<AutomationEntity.async_trigger() running at /usr/src/homeassistant/homeassistant/components/automation/init.py:673>> is being executed.

and

2024-04-05 00:41:19.016 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:19.020 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f391c003f10>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:19.022 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f391c26c0d0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:19.026 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f79dece0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-7982' coro=<ClientRequest.write_bytes() running at /usr/local/lib/python3.12/site-packages/aiohttp/client_reqrep.py:610> cb=[ClientRequest.__reset_writer(), ClientResponse.__reset_writer()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:19.027 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-311' coro=<AsyncioDispatcher.handle_timeout() running at /usr/local/lib/python3.12/site-packages/pysnmp/carrier/asyncio/dispatch.py:62> wait_for=> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:19.040 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-1790' coro=<Client._misc_loop() running at /usr/local/lib/python3.12/site-packages/aiomqtt/client.py:675> wait_for=> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:19.221 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) RuntimeError: Cannot enter into task <Task pending name='xbox - Home Assistant Cloud - refresh' coro=<DataUpdateCoordinator._handle_refresh_interval() running at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:258> wait_for= cb=[set.remove(), set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed.

2024-04-05 00:41:20.702 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='EntityPlatform poll media_player.androidtv' coro=<EntityPlatform._update_entity_states() running at /usr/src/homeassistant/homeassistant/helpers/entity_platform.py:1018> wait_for= cb=[set.remove(), set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:20.747 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='EntityPlatform poll media_player.androidtv' coro=<EntityPlatform._update_entity_states() running at /usr/src/homeassistant/homeassistant/helpers/entity_platform.py:1018> wait_for= cb=[set.remove(), set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:20.794 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task cancelling name='Task-7976' coro=<RemoteProtocol._async_idle_disconnect() running at /usr/local/lib/python3.12/site-packages/androidtvremote2/remote.py:203> wait_for=> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:20.796 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f7aab820>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-7986' coro=<RemoteProtocol._async_idle_disconnect() running at /usr/local/lib/python3.12/site-packages/androidtvremote2/remote.py:198>> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:20.935 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-253' coro=<Websocket._websocket_loop() running at /usr/local/lib/python3.12/site-packages/pyunifiprotect/websocket.py:103> wait_for=> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:20.939 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-261' coro=<Websocket._websocket_loop() running at /usr/local/lib/python3.12/site-packages/pyunifiprotect/websocket.py:103> wait_for=> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:21.167 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) RuntimeError: Cannot enter into task <Task pending name='active dings - - refresh' coro=<DataUpdateCoordinator._handle_refresh_interval() running at /usr/src/homeassistant/homeassistant/helpers/update_coordinator.py:258> wait_for= cb=[set.remove(), set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed.

2024-04-05 00:41:22.666 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task cancelling name='Task-7979' coro=<RemoteProtocol._async_idle_disconnect() running at /usr/local/lib/python3.12/site-packages/androidtvremote2/remote.py:203> wait_for=> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:22.669 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f3919f8a230>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-8033' coro=<RemoteProtocol._async_idle_disconnect() running at /usr/local/lib/python3.12/site-packages/androidtvremote2/remote.py:198>> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:22.698 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='plex websocket listener 35ccb4af885386dba83253b85e27381e' coro=<PlexWebsocket.listen() running at /usr/local/lib/python3.12/site-packages/plexwebsocket.py:234> wait_for= cb=[set.remove(), set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:23.602 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-355' coro=<WebSocketCommonProtocol.keepalive_ping() running at /usr/local/lib/python3.12/site-packages/websockets/legacy/protocol.py:1254> wait_for=> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:23.915 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_reported[L]: entity_id=media_player.playstation_4_2, old_last_reported=2024-04-05T00:41:13.915307+01:00, new_state=<state media_player.playstation_4_2=unknown; friendly_name=PlayStation 4, supported_features=6529 @ 2024-04-05T00:28:43.411959+01:00>> 2024-04-05 00:41:23.916 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_reported[L]: entity_id=media_player.playstation_4_2, old_last_reported=2024-04-05T00:41:23.915589+01:00, new_state=<state media_player.playstation_4_2=unknown; friendly_name=PlayStation 4, supported_features=6529 @ 2024-04-05T00:28:43.411959+01:00>> 2024-04-05 00:41:23.934 DEBUG (ArloBackgroundWorker) [pyaarlo] media-library: queueing image library update 2024-04-05 00:41:24.826 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-2403' coro=<RequestHandler.start() running at /usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py:505> wait_for=> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:24.828 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-2402' coro=<RequestHandler.start() running at /usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py:505> wait_for=> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:25.182 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f391c3ed990>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-8034' coro=<ClientRequest.write_bytes() running at /usr/local/lib/python3.12/site-packages/aiohttp/client_reqrep.py:610> cb=[ClientRequest.__reset_writer(), ClientResponse.__reset_writer()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:25.266 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='EntityPlatform poll switch.adguard' coro=<EntityPlatform._update_entity_states() running at /usr/src/homeassistant/homeassistant/helpers/entity_platform.py:1018> wait_for= cb=[set.remove(), set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:25.800 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending! 2024-04-05 00:41:25.802 WARNING (MainThread) [py.warnings] /usr/local/lib/python3.12/site-packages/androidtvremote2/remote.py:194: RuntimeWarning: coroutine 'RemoteProtocol._async_idle_disconnect' was never awaited self._idle_disconnect_task = self._loop.create_task(

2024-04-05 00:41:25.804 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f7a6a530>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) RuntimeError: Cannot enter into task <Task pending name='Task-8035' coro=<RemoteProtocol._async_idle_disconnect() running at /usr/local/lib/python3.12/site-packages/androidtvremote2/remote.py:198>> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed.

2024-04-05 00:41:40.376 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f39217e3310>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-8045' coro=<TCPConnector._resolve_host() running at /usr/local/lib/python3.12/site-packages/aiohttp/connector.py:825> cb=[shield.._inner_done_callback() at /usr/local/lib/python3.12/asyncio/tasks.py:905]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:40.404 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_reported[L]: entity_id=calendar.milton_keynes_council, old_last_reported=2024-04-05T00:40:40.403626+01:00, new_state=<state calendar.milton_keynes_council=off; message=140L Food & Garden (Green Lid), all_day=True, start_time=2024-04-09 00:00:00, end_time=2024-04-10 00:00:00, location=, description=, friendly_name=Milton Keynes council @ 2024-04-05T00:28:40.382574+01:00>> 2024-04-05 00:41:40.698 WARNING (MainThread) [homeassistant.components.media_player] Updating androidtv media_player took longer than the scheduled update interval 0:00:10 2024-04-05 00:41:40.701 WARNING (MainThread) [homeassistant.components.media_player] Updating androidtv media_player took longer than the scheduled update interval 0:00:10 2024-04-05 00:41:40.701 WARNING (MainThread) [homeassistant.components.media_player] Updating androidtv media_player took longer than the scheduled update interval 0:00:10 2024-04-05 00:41:40.823 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f3919eee860>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-8047' coro=<RemoteProtocol._async_idle_disconnect() running at /usr/local/lib/python3.12/site-packages/androidtvremote2/remote.py:198>> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:41.080 WARNING (MainThread) [py.warnings] /usr/local/lib/python3.12/site-packages/pyasn1/type/univ.py:2329: RuntimeWarning: coroutine 'RemoteProtocol._async_idle_disconnect' was never awaited def init(self, **kwargs):

2024-04-05 00:41:41.080 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending! 2024-04-05 00:41:41.080 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending! 2024-04-05 00:41:41.081 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending! 2024-04-05 00:41:41.081 WARNING (MainThread) [py.warnings] /usr/local/lib/python3.12/site-packages/pyasn1/type/univ.py:2329: RuntimeWarning: coroutine 'WebSocketWriter.ping' was never awaited def init(self, **kwargs):

2024-04-05 00:41:41.081 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending! 2024-04-05 00:41:41.081 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending! 2024-04-05 00:41:41.081 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending! 2024-04-05 00:41:41.082 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending! 2024-04-05 00:41:41.082 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending! 2024-04-05 00:41:41.082 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending!

2024-04-05 00:41:42.444 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f6110280>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:41:42.740 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending! 2024-04-05 00:41:42.741 WARNING (MainThread) [py.warnings] /usr/local/lib/python3.12/site-packages/androidtvremote2/remote.py:194: RuntimeWarning: coroutine 'RemoteProtocol._async_idle_disconnect' was never awaited self._idle_disconnect_task = self._loop.create_task(

2024-04-05 00:41:42.743 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f39445ca230>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) RuntimeError: Cannot enter into task <Task pending name='Task-8117' coro=<RemoteProtocol._async_idle_disconnect() running at /usr/local/lib/python3.12/site-packages/androidtvremote2/remote.py:198>> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed.

2024-04-05 00:42:03.338 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f61ddf30>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:03.339 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f61dc5e0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:03.341 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f61dd420>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:03.343 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f61de4d0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:03.344 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f61dda80>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:03.345 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f61dc6d0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed.

2024-04-05 00:42:10.269 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f618feb0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:10.270 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f61dd720>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:10.272 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'occupied_heating_setpoint' when rendering '{{ value_json.occupied_heating_setpoint }}' 2024-04-05 00:42:10.273 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sensor.nursery_thermostat_pi_heating_demand, old_state=<state sensor.nursery_thermostat_pi_heating_demand=unknown; action=None, battery=100, humidity=58.97, keypad_lockout=None, last_seen=2024-04-05T00:42:10+01:00, linkquality=65, local_temperature=20.44, pi_heating_demand=None, voltage=3000, unit_of_measurement=%, icon=mdi:radiator, friendly_name=Nursery Thermostat PI heating demand @ 2024-04-05T00:28:43.878374+01:00>, new_state=<state sensor.nursery_thermostat_pi_heating_demand=unknown; action=None, battery=100, humidity=58.97, keypad_lockout=None, last_seen=2024-04-05T00:42:10+01:00, linkquality=61, local_temperature=20.44, pi_heating_demand=None, voltage=3000, unit_of_measurement=%, icon=mdi:radiator, friendly_name=Nursery Thermostat PI heating demand @ 2024-04-05T00:28:43.878374+01:00>> 2024-04-05 00:42:10.274 DEBUG (MainThread) [homeassistant.components.automation.vacuum_update_map_extractor_camera] Automation trigger 'None' triggered by event 'state_changed' 2024-04-05 00:42:10.281 WARNING (MainThread) [py.warnings] /usr/local/lib/python3.12/site-packages/jinja2/filters.py:1733: RuntimeWarning: coroutine 'BaseSelectorEventLoop._accept_connection2' was never awaited def prepare_select_or_reject(

2024-04-05 00:42:10.281 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending! 2024-04-05 00:42:10.282 ERROR (MainThread) [homeassistant] Error doing job: Task was destroyed but it is pending!

2024-04-05 00:42:10.336 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f61df910>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:10.337 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f61df4f0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:10.338 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f61ddc90>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:10.339 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38fe13c640>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:10.339 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38fe13cd90>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:10.340 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f391c2de9b0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:10.341 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f61de500>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:10.341 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f61df400>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:10.342 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38fe13c790>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:10.343 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38fe13c610>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed.

2024-04-05 00:42:10.914 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f774bd90>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) RuntimeError: Cannot enter into task <Task pending name='Task-8182' coro=<RemoteProtocol._async_idle_disconnect() running at /usr/local/lib/python3.12/site-packages/androidtvremote2/remote.py:198>> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:11.464 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_reported[L]: entity_id=media_player.tv_ue55ju7000, old_last_reported=2024-04-05T00:42:01.462896+01:00, new_state=<state media_player.tv_ue55ju7000=unavailable; friendly_name=[TV] UE55JU7000, supported_features=0 @ 2024-04-05T00:28:41.318763+01:00>> 2024-04-05 00:42:11.554 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_reported[L]: entity_id=sensor.octopus_energy_a_82e6a672_octoplus_points, old_last_reported=2024-04-05T00:41:41.552780+01:00, new_state=<state sensor.octopus_energy_a_82e6a672_octoplus_points=1844; state_class=total, last_evaluated=2024-04-05T00:29:11.400853+01:00, data_last_retrieved=2024-04-05T00:29:11.400853+01:00, icon=mdi:trophy, friendly_name=Octopus Energy A-82E6A672 Octoplus Points @ 2024-04-05T00:28:41.380118+01:00>>

2024-04-05 00:42:18.213 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38fe13e3b0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:19.056 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-7645' coro=<RequestHandler.start() running at /usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py:505> wait_for=> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:19.420 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) RuntimeError: Cannot enter into task <Task pending name='Task-6972' coro=<RequestHandler.start() running at /usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py:505> wait_for=> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed.

2024-04-05 00:42:20.103 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f391c26dc90>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:20.702 WARNING (MainThread) [homeassistant.components.media_player] Updating androidtv media_player took longer than the scheduled update interval 0:00:10 2024-04-05 00:42:20.704 WARNING (MainThread) [homeassistant.components.media_player] Updating androidtv media_player took longer than the scheduled update interval 0:00:10 2024-04-05 00:42:20.704 WARNING (MainThread) [homeassistant.components.media_player] Updating androidtv media_player took longer than the scheduled update interval 0:00:10 2024-04-05 00:42:20.925 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f39197853f0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-8254' coro=<RemoteProtocol._async_idle_disconnect() running at /usr/local/lib/python3.12/site-packages/androidtvremote2/remote.py:198>> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed.

2024-04-05 00:42:21.554 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f391bc4d810>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-8256' coro=<BaseSelectorEventLoop._accept_connection2() running at /usr/local/lib/python3.12/asyncio/selector_events.py:213>> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:21.558 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f39217e73a0>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='Task-8257' coro=<BaseSelectorEventLoop._accept_connection2() running at /usr/local/lib/python3.12/asyncio/selector_events.py:213>> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed.

2024-04-05 00:42:38.952 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f6111480>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:40.405 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_reported[L]: entity_id=calendar.milton_keynes_council, old_last_reported=2024-04-05T00:41:40.404468+01:00, new_state=<state calendar.milton_keynes_council=off; message=140L Food & Garden (Green Lid), all_day=True, start_time=2024-04-09 00:00:00, end_time=2024-04-10 00:00:00, location=, description=, friendly_name=Milton Keynes council @ 2024-04-05T00:28:40.382574+01:00>> 2024-04-05 00:42:40.662 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup(<Future finis....242, ttl=60]>) Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='EntityPlatform poll sensor.dnsip' coro=<EntityPlatform._update_entity_states() running at /usr/src/homeassistant/homeassistant/helpers/entity_platform.py:1018> wait_for=<Future finished result=[<ares_query_a...45.242, ttl=60]> cb=[set.remove(), set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed.

2024-04-05 00:42:38.952 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f38f6111480>() Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers..async_on_state_changed() running at /config/custom_components/watchman/init.py:303> cb=[set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed. 2024-04-05 00:42:40.405 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_reported[L]: entity_id=calendar.milton_keynes_council, old_last_reported=2024-04-05T00:41:40.404468+01:00, new_state=<state calendar.milton_keynes_council=off; message=140L Food & Garden (Green Lid), all_day=True, start_time=2024-04-09 00:00:00, end_time=2024-04-10 00:00:00, location=, description=, friendly_name=Milton Keynes council @ 2024-04-05T00:28:40.382574+01:00>> 2024-04-05 00:42:40.662 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup(<Future finis....242, ttl=60]>) Traceback (most recent call last): File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, self._args) RuntimeError: Cannot enter into task <Task pending name='EntityPlatform poll sensor.dnsip' coro=<EntityPlatform._update_entity_states() running at /usr/src/homeassistant/homeassistant/helpers/entity_platform.py:1018> wait_for=<Future finished result=[<ares_query_a...45.242, ttl=60]> cb=[set.remove(), set.remove()]> while another task <Task pending name='Task-6968' coro=<WebSocketHandler._writer() running at /usr/src/homeassistant/homeassistant/components/websocket_api/http.py:138> wait_for=> is being executed.

and then the webui just dies at one point while still a few logs are being written and ha then restarts ... not sure if any of the above makes any sense

jezzamine commented 8 months ago

Same log errors with 2024.4.0. I don't get these errors when I disable the hass-aarlo custom integration. I've opened an issue at []https://github.com/twrecked/hass-aarlo/issues/907 if anyone would like to add more info.

collse commented 8 months ago

Same log errors with 2024.4.0. I don't get these errors when I disable the hass-aarlo custom integration. I've opened an issue at []twrecked/hass-aarlo#907 if anyone would like to add more info.

Which errors are you referring to in particular?

edit: read your report - they differ to mine though

jezzamine commented 8 months ago

I have a lot of the same RuntimeError errors as you reported. They start appearing (and quickly multiplying!) after the inital errors in my report.

kotope commented 8 months ago

I've also noticed random crashes since update to 2024.4.0 that didn't happen with 2024.3.3.

Already reverted back to older version and have did not check the logs from 2024.4. Will try latest release again during the weekend and see the logs.

collse commented 8 months ago

my instance is running for 30+ minutes now with watchman + aarlo disabled no crashes so far - it usually died in 15-20

gustafssone commented 8 months ago

Same here, crashes after an hour or so. Didn't have any issues in 2024.3.3. Running in container, no changes really in the yaml or so that I've done.

Constantly in the logs is: 2024-04-05T10:18:53.329857453Z RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers.<locals>.async_on_state_changed() running at /config/custom_components/watchman/__init__.py:303> cb=[set.remove()]> while another task <Task pending name='anyio._core._sockets.connect_tcp.<locals>.try_connect' coro=<connect_tcp.<locals>.try_connect() running at /usr/local/lib/python3.12/site-packages/anyio/_core/_sockets.py:170> wait_for=<Future finished result=None> cb=[TaskGroup._spawn.<locals>.task_done() at /usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py:699]> is being executed. 2024-04-05T10:18:53.329853448Z File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run 2024-04-05T10:18:53.329850863Z Traceback (most recent call last): 2024-04-05T10:18:53.329855508Z self._context.run(self._callback, *self._args) 2024-04-05T10:18:53.983152125Z 2024-04-05 12:15:56.361 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback <_asyncio.TaskStepMethWrapper object at 0x7f1c1b784c10>() 2024-04-05T10:18:53.983179743Z Traceback (most recent call last): 2024-04-05T10:18:53.983182538Z File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run 2024-04-05T10:18:53.983185601Z self._context.run(self._callback, *self._args) 2024-04-05T10:18:53.983188509Z RuntimeError: Cannot enter into task <Task pending name='listen state_changed' coro=<add_event_handlers.<locals>.async_on_state_changed() running at /config/custom_components/watchman/__init__.py:303> cb=[set.remove()]> while another task <Task pending name='anyio._core._sockets.connect_tcp.<locals>.try_connect' coro=<connect_tcp.<locals>.try_connect() running at /usr/local/lib/python3.12/site-packages/anyio/_core/_sockets.py:170> wait_for=<Future finished result=None> cb=[TaskGroup._spawn.<locals>.task_done() at /usr/local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py:699]> is being executed.

kotope commented 8 months ago

Another crash during night at 5:03 (GMT+3) with 2024.4.1.

I also had Watchman installed and thought it was the problem, so I uninstalled it. Did not help.

Anyhow, getting tons of following log errors about five minutes prior the crash from various integrations:

...
RuntimeError: Cannot enter into task <Task cancelling name='Task-1947720' coro=<RequestHandler.start() running at /usr/local/lib/python3.12/site-packages/aiohttp/web_protocol.py:505> wait_for=<Future cancelled>> while another task <Task pending name='eink-dashboard @ 192.168.1.203: aioesphomeapi connect' coro=<ReconnectLogic._connect_once_or_reschedule() running at /usr/local/lib/python3.12/site-packages/aioesphomeapi/reconnect_logic.py:298> wait_for=<Future finished exception=OSError(113, "Connect call failed ('192.168.1.203', 6053)")>> is being executed.
2024-04-06 04:57:46.212 ERROR (SyncWorker_2) [homeassistant] Error doing job: Task was destroyed but it is pending!
2024-04-06 04:57:46.212 ERROR (SyncWorker_2) [homeassistant] Error doing job: Task was destroyed but it is pending!
2024-04-06 04:57:46.421 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback Task.task_wakeup(<Future finis... 11.975, ...}>)
...

For me it looks like something is getting stuck or memory running out?

Running Home Assistant supervised on Proxmox VM.

If you need the full logs, let me know and I'll provide them for you. Rather not paste the full logs in here in case of possible sensitive information.

bdraco commented 8 months ago

Most of these logs look like integrations doing non-threadsafe operations on asyncio objects which is corrupting the internal asyncio state.

When an integration modifies the event loop internals from another thread the behavior is undefined and will usually eventually lead to a crash.

If you put asyncio in debug mode, it will be able to detect some of these bad calls and maybe even block them. You can use the profiler service to do that: https://www.home-assistant.io/integrations/profiler/#service-profilerset_asyncio_debug

If you can't get far enough along you can also enable debugpy: in configuration.yaml instead (but its much slower) https://www.home-assistant.io/integrations/debugpy/

kotope commented 8 months ago

Thanks @bdraco !

Enabled the debugging at seemed like async queue was flooded by couple of integrations on my environment:

Just disabled those integrations for now to see if I still get crashes.

collse commented 8 months ago

Most of these logs look like integrations doing non-threadsafe operations on asyncio objects which is corrupting the internal asyncio state.

When an integration modifies the event loop internals from another thread the behavior is undefined and will usually eventually lead to a crash.

If you put asyncio in debug mode, it will be able to detect some of these bad calls and maybe even block them. You can use the profiler service to do that: https://www.home-assistant.io/integrations/profiler/#service-profilerset_asyncio_debug

If you can't get far enough along you can also enable debugpy: in configuration.yaml instead (but its much slower) https://www.home-assistant.io/integrations/debugpy/

tried the profiler with its link, there is no such service call: Service profiler.set_asyncio_debug not found.

smoki3 commented 8 months ago

So there are many reports in 2024.4 that it chrashes:

https://github.com/home-assistant/core/issues/114958

https://github.com/home-assistant/core/issues/114944

https://github.com/home-assistant/core/issues/115007

kotope commented 8 months ago

Most of these logs look like integrations doing non-threadsafe operations on asyncio objects which is corrupting the internal asyncio state. When an integration modifies the event loop internals from another thread the behavior is undefined and will usually eventually lead to a crash. If you put asyncio in debug mode, it will be able to detect some of these bad calls and maybe even block them. You can use the profiler service to do that: https://www.home-assistant.io/integrations/profiler/#service-profilerset_asyncio_debug If you can't get far enough along you can also enable debugpy: in configuration.yaml instead (but its much slower) https://www.home-assistant.io/integrations/debugpy/

tried the profiler with its link, there is no such service call: Service profiler.set_asyncio_debug not found.

Add 'profiler' integration from the integrations page first.

collse commented 8 months ago

should have read the entire article - thanks @kotope

collse commented 8 months ago

there is no consistency to it - the combination of custom_components that what worked yesterday with 2024.4.0 no longer does with 2024.4.1 and it crashes even faster ... had to disable 2 completely now to see what is happening

bdraco commented 8 months ago

there is no consistency to it

Remember that core devs do not review custom components, so they are never checked for blocking I/O in the event loop or performing non-thread-safe operations. It's likely that you have more than one that has a problem, and it may be rare enough that it's only hit it sometimes. While these bugs have likely existed for a long time in the custom components, since 2024.4.x is a lot faster, you are likely to hit them quickly and thus crash faster.

https://docs.python.org/3/library/asyncio-dev.html#concurrency-and-multithreading

Your logs in https://github.com/twrecked/hass-aarlo/issues/907#issuecomment-2041050228 show where the bug is in aarlo as its clearly calling the async api from a thread which is not thread safe RuntimeError: Non-thread-safe operation invoked on an event loop other than the current one

  File "/config/custom_components/aarlo/camera.py", line 410, in update_state
    self.async_schedule_update_ha_state()

https://github.com/twrecked/hass-aarlo/issues/907#issuecomment-2041143921 https://github.com/twrecked/hass-aarlo/issues/907#issuecomment-2041144520

maikrovers commented 7 months ago

Same issue after the update. It runs for 30 minutes or so, after that the GUI dies. Going to try to disable Aarlo integration. Too bad, because I use the motion sensors from Aarlo.

marcbecker1983 commented 7 months ago

Same issue here but sometimes it crashes after some minutes and sometimes it runs more than one hour. This issue happens with the latest HA versions 2024.4.x - 2024.3.3 was running super stable.

maikrovers commented 7 months ago

Same issue here but sometimes it crashes after some minutes and sometimes it runs more than one hour. This issue happens with the latest HA versions 2024.4.x - 2024.3.3 was running super stable.

Yes, I will go on and try the Beta verion of the Aarlo integration: 0.8.1b14 Fix callback threading issues.

Fix is released 3 days ago...

marcbecker1983 commented 7 months ago

Same issue here but sometimes it crashes after some minutes and sometimes it runs more than one hour. This issue happens with the latest HA versions 2024.4.x - 2024.3.3 was running super stable.

Yes, I will go on and try the Beta verion of the Aarlo integration: 0.8.1b14 Fix callback threading issues.

Fix is released 3 days ago...

I think not just only the Aarlo integration is causing this issue because I don't use the integration on my machine

dagstuan commented 7 months ago

Chiming in with the same issues. Semi-frequent hangs after updating to 2024.4. No issues with 2024.3. No apparent errors in the logs when the crashes happen unfortunately... These are the only relevant logs from the last crash. HA crashed at 00:05 according to my healthcheck. I'm not running the Aarlo integration others are mentioning.

2024-04-15 00:09:02.312 WARNING (Thread-2 (_thread_main)) [homeassistant.components.mqtt.client] Disconnected from MQTT server mqtt:1883 (7)
2024-04-15 00:14:41.350 WARNING (Thread-2 (_thread_main)) [homeassistant.components.mqtt.client] Disconnected from MQTT server mqtt:1883 (7)
2024-04-15 00:25:45.134 WARNING (Thread-2 (_thread_main)) [homeassistant.components.mqtt.client] Disconnected from MQTT server mqtt:1883 (7)
2024-04-15 00:28:17.760 WARNING (MainThread) [homeassistant.helpers.entity] Updating state for climate.bad (<class 'homeassistant.components.mqtt.climate.MqttClimate'>) took 0.915 seconds. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+mqtt%22
2024-04-15 01:05:52.806 WARNING (Thread-2 (_thread_main)) [homeassistant.components.mqtt.client] Disconnected from MQTT server mqtt:1883 (7)
2024-04-15 01:38:57.897 WARNING (Thread-2 (_thread_main)) [homeassistant.components.mqtt.client] Disconnected from MQTT server mqtt:1883 (7)
2024-04-15 01:40:09.164 ERROR (MainThread) [homeassistant.components.mill] Timeout fetching mill data
2024-04-15 01:40:11.816 ERROR (MainThread) [homeassistant.components.mill] Timeout fetching mill data
2024-04-15 01:40:38.695 ERROR (MainThread) [homeassistant.components.recorder.core] The recorder backlog queue reached the maximum size of 73843 events; usually, the system is CPU bound, I/O bound, or the database is corrupt due to a disk problem; The recorder will stop recording events to avoid running out of memory

Restarting the Docker-container makes HA come up again. Fault logs are empty unfortunately. It seems to be more of a "hang" than an actual crash.

bdraco commented 7 months ago

It seems to be more of a "hang" than an actual crash.

Please start a fresh issue report as your instance has a different problem and will have a different path to resolution

gustafssone commented 7 months ago

In my case it appears to be https://github.com/greghesp/ha-bambulab which had some async issues. I have enabled the debug on asyncio and so far no crash as I have applied a beta of that custom_component (see thread and comment at) https://github.com/greghesp/ha-bambulab/issues/479#issuecomment-2058331070 . Now 2024.4 seems to be stable. HA 2024.3 and ha-bambulab v2.0.15 had no issues for me. HA 2024.4 and ha-bambulab v2.0.15 crashed. HA 2024.4 and ha-bambulab v2.0.16-dev4 no issues (so far).

issue-triage-workflows[bot] commented 7 months ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment πŸ‘ This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

wolph commented 7 months ago

I've been experiencing this issue since the last release as well. HA is regularly restarting (I'm assuming because of the health check) and it's quite annoying. There are few logs before the restarts but nothing all too useful in tracking it down.

I don't have any bambulabs stuff so I can rule that out, in my cases I do see regular warnings about the bluetooth scanner going quiet and a warning from a pychromecast heartbeat timeout.

bdraco commented 7 months ago

https://community.home-assistant.io/t/2024-5-tracking-down-instability-issues-caused-by-integrations/724441

r1si commented 7 months ago

I have the SAME bug from yesterday after update to 2024.4.3...

I think that after the latest update HA has become more sensitive to loop problems that can cause custom integration

I for one think it may be the Dreame integration....

wolph commented 7 months ago

As a temporary(?) workaround you can make your healthcheck rules more relaxed so it won't get killed anymore. But seeing how often this is happening I'm certain that some parts of Home Assistant have become slower and/or blocking which cause the healthcheck to fail.

"signal 9" is a kill signal so something external killed home-assistant

issue-triage-workflows[bot] commented 6 months ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment πŸ‘ This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

wolph commented 6 months ago

I've found the issue for my installation. In addition to the healtcheck killing it because it was responding too slow at times, it was also running out of memory. It seems that the latest Home Assistant version uses significantly more memory than the old versions (I had the limit set to 1 GiB and previously it would top out at about 600-700 MiB) but it would go over the 1 GiB with the current release and get killed by the OOM (Out Of Memory) killer.

bdraco commented 6 months ago

There is sometimes a strong correlation between memory usage and sluggish performance. You might be able to find the source with https://community.home-assistant.io/t/2024-5-tracking-down-instability-issues-caused-by-integrations/724441

Sluggish performance or increased CPU usage

issue-triage-workflows[bot] commented 6 months ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment πŸ‘ This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.