search

ateodorescu / home-assistant-ipmi

IPMI connector for Home Assistant
MIT License
39 stars 10 forks source link

Slow integration start and power sensors not available when restarting the HA system (intermittent issue). #35

Closed 35gh closed 3 months ago

35gh commented 5 months ago

Frequently when I restart the HA system (full system restart) or when I do a simple HA restart (less frequent) I see those warnings in logs:

Enregistreur: homeassistant.bootstrap
Source: bootstrap.py:661
S'est produit pour la première fois: 15:47:21 (1 occurrences)
Dernier enregistrement: 15:47:21
Waiting on integrations to complete setup: {('ipmi', 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'): 31.759796395}
Enregistreur: homeassistant.components.sensor
Source: runner.py:189
intégration: Capteur (documentation, problèmes)
S'est produit pour la première fois: 15:47:34 (1 occurrences)
Dernier enregistrement: 15:47:34
Setup of sensor platform ipmi is taking over 10 seconds.

Followed from time to time by this error (not always):

Enregistreur: custom_components.ipmi
Source: helpers/update_coordinator.py:321
intégration: IPMI connector (documentation, problèmes)
S'est produit pour la première fois: 15:48:24 (1 occurrences)
Dernier enregistrement: 15:48:24
Timeout fetching IPMI coordinator data

Then it fails with:

Enregistreur: homeassistant
Source: util/async_.py:35
S'est produit pour la première fois: 15:50:37 (1 occurrences)
Dernier enregistrement: 15:50:37
Error doing job: Task exception was never retrieved

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 629, in async_add_entities
    await add_func(coros, entities, timeout)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 535, in _async_add_and_update_entities
    tasks = [create_eager_task(coro) for coro in coros]
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 35, in create_eager_task
    loop=loop or get_running_loop(),
                 ^^^^^^^^^^^^^^^^^^
RuntimeError: no running event loop

As a consequence, power sensors are not available (other entities are ok):

Capture d’écran 2024-04-09 à 09 44 57

No error in IPMItool server logs:

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service base-addon-banner: starting

-----------------------------------------------------------
 Add-on: IPMItool server
 IPMItool server
-----------------------------------------------------------
 Add-on version: 1.2.9
 You are running the latest version of this add-on.
 System: Home Assistant OS 12.2  (amd64 / qemux86-64)
 Home Assistant Core: 2024.4.3
 Home Assistant Supervisor: 2024.04.0
-----------------------------------------------------------
 Please, share the above information when looking for help
 or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
s6-rc: info: service base-addon-banner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service base-addon-log-level: starting
s6-rc: info: service fix-attrs successfully started
Log level is set to TRACE
s6-rc: info: service base-addon-log-level successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun nginx (no readiness notification)
services-up: info: copying legacy longrun php-fpm (no readiness notification)
[15:47:31] TRACE: bashio::net.wait_for 9000
[15:47:31] INFO: Starting PHP-FPM...
s6-rc: info: service legacy-services successfully started
[15:47:31] INFO: Starting NGinx...

A simple HA restart usually fixes the issue.

HA: 2024.4.3 IPMI connector: v1.6.0 IPMItool server: v1.2.9 Motherboard: Supermicro

Thank you!

MNeverOff commented 5 months ago

Since the move to the new way of registering components (as far as I was able to understand the 1.5.0 change) I have been experiencing the same. The fact that my IPMI interface isn't the fastest and can sometimes take over 50 seconds doesn't help either.

I re-packaged the ipmi-sever to increase tolerance and timeouts but it's sometimes very difficult / impossible to get that running during startup. Restarts still do seem to be fixing it, my error is close:

│   [31m2024-04-14 23:46:12.600 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry ipmi@192.168.1.1:623 for ipmi                                      │
│   Traceback (most recent call last):                                                                                                                                              │
│     File "/usr/src/homeassistant/homeassistant/config_entries.py", line 551, in async_setup                                                                                       │
│       result = await component.async_setup_entry(hass, self)                                                                                                                      │
│                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                      │
│     File "/config/custom_components/ipmi/__init__.py", line 127, in async_setup_entry                                                                                             │
│       await coordinator.async_config_entry_first_refresh()                                                                                                                        │
│     File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 280, in async_config_entry_first_refresh                                                      │
│       await self._async_refresh(                                                                                                                                                  │
│     File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 315, in _async_refresh                                                                        │
│       self.data = await self._async_update_data()                                                                                                                                 │
│                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                 │
│     File "/config/custom_components/ipmi/__init__.py", line 223, in _async_update_data                                                                                            │
│       await self.hass.async_add_executor_job(self.ipmiData.update)                                                                                                                │
│   asyncio.exceptions.CancelledError[0m
ateodorescu commented 5 months ago

I'm sorry but I have absolutely no idea why this fails:

Enregistreur: homeassistant
Source: util/async_.py:35
S'est produit pour la première fois: 15:50:37 (1 occurrences)
Dernier enregistrement: 15:50:37
Error doing job: Task exception was never retrieved

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 629, in async_add_entities
    await add_func(coros, entities, timeout)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 535, in _async_add_and_update_entities
    tasks = [create_eager_task(coro) for coro in coros]
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 35, in create_eager_task
    loop=loop or get_running_loop(),
                 ^^^^^^^^^^^^^^^^^^
RuntimeError: no running event loop

I do not understand why it fails when I add the entities in a listener using the callback that the API provided. I do not know if there is something that I can fix or it's something that the core team has to fix in the HASS API. I just know that it does not work to add entities in the listener: https://github.com/ateodorescu/home-assistant-ipmi/blob/c8576bcf652c4dc84261c098c135c121c57a5764/custom_components/ipmi/sensor.py#L101

ateodorescu commented 3 months ago

Does this still happen with latest version?

35gh commented 3 months ago

It seems behaviour has improved in latest HA version (using Core 2024.6.4 / Supervisor 2024.06.2 / Operating System 12.4).

After restart I get from time to time a delayed start message:

Enregistreur: homeassistant.bootstrap
Source: bootstrap.py:726
S'est produit pour la première fois: 10:55:32 (1 occurrences)
Dernier enregistrement: 10:55:32
Waiting on integrations to complete setup: {('ipmi', '279973283e5512c52af8e8f2f97d15da'): 34.933692502}

but after a while IPMI sensors are coming up including power sensors, with no additional warning/error in logs.

ateodorescu commented 3 months ago

It seems behaviour has improved in latest HA version (using Core 2024.6.4 / Supervisor 2024.06.2 / Operating System 12.4).

After restart I get from time to time a delayed start message:

Enregistreur: homeassistant.bootstrap
Source: bootstrap.py:726
S'est produit pour la première fois: 10:55:32 (1 occurrences)
Dernier enregistrement: 10:55:32
Waiting on integrations to complete setup: {('ipmi', '279973283e5512c52af8e8f2f97d15da'): 34.933692502}

but after a while IPMI sensors are coming up including power sensors, with no additional warning/error in logs.

do you use the latest version of ipmi integration and ipmi-server addon?

35gh commented 3 months ago

yes - IPMI Connector v1.7.0 and IPMItool server v1.2.9