Names and device

[kifort]

First of all: thanks for your great project. In "both" mode, I see 78 wemportal entities and I am bit lost between them. Some of them are in german, some in english due to issue #22. Some of them have a prefix of "heat_pump", "heating_circuits" or "wez", others have none of these. Could you maybe add a "Heat Pump" device? That would allow to add all entities to an area in one step. Also all sensors or all controls could be added to a dashboard in one step. Otherwise, I suggest to add some common entity prefixes that can be used in entity filtering of the lovelace cards. Something like:

• wem_
• wem_web
• wem_api

[dm82m]

We are using the default names we get from wemportal api or Web. If you want you can just go ahead and rename your entities however you prefer.

[erikkastelec]

Fixing naming is quite hard in this case. By dynamically combining data from API and website there are a lot of naming conflicts, which can't be resolved with simple logic. To get everything fixed I would probably need to hard code a lot of things. There is also the problem with different devices having different entities and settings, which complicates this further. For now, I will focus on improving the functionality and leave the naming when everything else works sufficiently well.

Some of them have a prefix of "heat_pump", "heating_circuits" or "wez", others have none of these.

Prefixes are used for entities that are scraped from the WEM portal website. Prefixes are the categories that can be seen on the website and are there to help you narrow down what the entity name actually means.

Could you maybe add a "Heat Pump" device?

This sounds like a good idea. Will add this, but can't provide an accurate time frame.

Otherwise, I suggest to add some common entity prefixes that can be used in entity filtering of the lovelace cards. Something like:

I think that Home Assistant developers discourage this. Grouping the entities into a device should solve this anyway.

[kifort]

Thank you both for the quick answer. I fully agree, that device is the right thing. Once it comes naming won't be a big issue anymore.

"Could you maybe add a "Heat Pump" device? This sounds like a good idea. Will add this, but can't provide an accurate time frame."

Should I leave this ticket open for "Heat Pump" device, or close it and create a new one?

[erikkastelec]

You can leave it open

[erikkastelec]

Device support is finally here and should be released soon after I test it out.

If anyone is interested in testing:

• Put all the files from /custom_conponents/wemportal/ that are in dev branch into homeassistat_config/custom_components/wemportal folder on your server.
• Add new integration inside Home Assistant frontend (yaml support is deprecated)

[dm82m]

Is there anything we need to know if we Test it. Any migration steps needed? Or just straight forward?

[erikkastelec]

It will probably brake names of entities ( append 2 at the end because entity with that name already exists). Other than that everything should work fine.

[dm82m]

And will brake history aswell, right? Maybe we should think about an entity migration so that people are not losing their history!?

[dm82m]

Maybe something like we did here: https://github.com/dm82m/hass-Deltasol-KM2/commit/9a500f342a3d792bb8c705cd7708e0330e40964b

[erikkastelec]

Looks good, will add this.

[dm82m]

migration seems not to work, I tried the dev branch on a clean install. started with current released version and then migrated to dev branch. old sensors will get state "restored" and new sensors are created. maybe this has something to do with changing from yaml config to config flow? not sure tbh.

I also get no info logs about entity migration, I guess something is not good there.

[erikkastelec]

Interesting, I did the same and it worked. Will spin up a new instance, see if I can reproduce this and keep you posted.

[erikkastelec]

It should work now. After adding the integration, be sure to configure any additional settings in the GUI, as the settings in the configuration file no longer have any effect.

Thank you for taking the time to test this.

[dm82m]

2023-03-10 10:46:05.712 WARNING (MainThread) [homeassistant.helpers.frame] Detected integration that uses deprecatedasync_get_registryto access entity registry, use async_get instead. Please report issue to the custom integration author for wemportal using this method at custom_components/wemportal/__init__.py, line 46: er = await entity_registry.async_get_registry(hass)

[dm82m]

2023-03-10 10:45:19.629 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration wemportal which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-03-10 10:45:47.479 INFO (MainThread) [homeassistant.setup] Setting up wemportal
2023-03-10 10:45:47.479 INFO (MainThread) [homeassistant.setup] Setup of domain wemportal took 0.0 seconds
2023-03-10 10:45:47.612 DEBUG (SyncWorker_2) [custom_components.wemportal] Fetching api device data
2023-03-10 10:45:47.761 DEBUG (SyncWorker_2) [custom_components.wemportal] Fetching api parameters data
2023-03-10 10:45:52.012 DEBUG (SyncWorker_2) [custom_components.wemportal] Fetching fresh api data
2023-03-10 10:46:05.707 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 18.227 seconds (success: True)
2023-03-10 10:46:05.707 INFO (MainThread) [custom_components.wemportal] Migrating entity names for wemportal
2023-03-10 10:46:05.712 WARNING (MainThread) [homeassistant.helpers.frame] Detected integration that uses deprecated `async_get_registry` to access entity registry, use async_get instead. Please report issue to the custom integration author for wemportal using this method at custom_components/wemportal/__init__.py, line 46: er = await entity_registry.async_get_registry(hass)
2023-03-10 10:46:05.715 INFO (MainThread) [custom_components.wemportal] Found entity with old id (select.hzk1_u_raumsoll). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK1-U_Raumsoll).
2023-03-10 10:46:05.715 INFO (MainThread) [custom_components.wemportal] Found entity with old id (number.hzk1_komfort). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK1-Komfort).
2023-03-10 10:46:05.716 INFO (MainThread) [custom_components.wemportal] Found entity with old id (number.hzk1_normal). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK1-Normal).
2023-03-10 10:46:05.716 INFO (MainThread) [custom_components.wemportal] Found entity with old id (number.hzk1_absenk). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK1-Absenk).
2023-03-10 10:46:05.717 INFO (MainThread) [custom_components.wemportal] Found entity with old id (select.hzk1_u_funktion). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK1-U_Funktion).
2023-03-10 10:46:05.717 INFO (MainThread) [custom_components.wemportal] Found entity with old id (select.hzk1_pp_funktion). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK1-PP_Funktion).
2023-03-10 10:46:05.718 INFO (MainThread) [custom_components.wemportal] Found entity with old id (select.hzk1_pp_raumsoll). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK1-PP_Raumsoll).
2023-03-10 10:46:05.718 INFO (MainThread) [custom_components.wemportal] Found entity with old id (select.hzk1_betriebsart). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK1-Betriebsart).
2023-03-10 10:46:05.719 INFO (MainThread) [custom_components.wemportal] Found entity with old id (select.hzk0_betriebsart). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK0-Betriebsart).
2023-03-10 10:46:05.719 INFO (MainThread) [custom_components.wemportal] Found entity with old id (select.hzk0_pp_funktion). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK0-PP_Funktion).
2023-03-10 10:46:05.719 INFO (MainThread) [custom_components.wemportal] Found entity with old id (number.hzk0_normal). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK0-Normal).
2023-03-10 10:46:05.725 INFO (MainThread) [custom_components.wemportal] Found entity with old id (number.hzk0_absenk). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK0-Absenk).
2023-03-10 10:46:05.726 INFO (MainThread) [custom_components.wemportal] Found entity with old id (select.hzk0_u_funktion). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK0-U_Funktion).
2023-03-10 10:46:05.726 INFO (MainThread) [custom_components.wemportal] Found entity with old id (select.hzk0_u_raumsoll). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK0-U_Raumsoll).
2023-03-10 10:46:05.726 INFO (MainThread) [custom_components.wemportal] Found entity with old id (select.hzk0_pp_raumsoll). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK0-PP_Raumsoll).
2023-03-10 10:46:05.726 INFO (MainThread) [custom_components.wemportal] Found entity with old id (number.hzk0_komfort). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:HZK0-Komfort).
2023-03-10 10:46:05.727 INFO (MainThread) [custom_components.wemportal] Found entity with old id (number.ww0_normalww). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:WW0-NormalWW).
2023-03-10 10:46:05.727 INFO (MainThread) [custom_components.wemportal] Found entity with old id (number.ww0_absenkww). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:WW0-AbsenkWW).
2023-03-10 10:46:05.727 INFO (MainThread) [custom_components.wemportal] Found entity with old id (switch.ww0_ww_push). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:WW0-WW-Push).
2023-03-10 10:46:05.728 INFO (MainThread) [custom_components.wemportal] Found entity with old id (number.we0_ww_solltemp). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:WE0-WW-Solltemp).
2023-03-10 10:46:05.730 DEBUG (SyncWorker_1) [custom_components.wemportal] Fetching fresh api data
2023-03-10 10:46:17.169 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 11.441 seconds (success: True)
2023-03-10 10:46:17.176 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.wemportal
2023-03-10 10:46:17.178 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.wemportal entity: sensor.hzk1_aktraumsoll
2023-03-10 10:46:17.179 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.wemportal entity: sensor.hzk1_raumtemperatur
2023-03-10 10:46:17.179 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.wemportal entity: sensor.hzk0_raumtemperatur
2023-03-10 10:46:17.180 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.wemportal entity: sensor.hzk0_aktraumsoll
2023-03-10 10:46:17.181 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.wemportal entity: sensor.system0_aussentemperatur
2023-03-10 10:46:17.181 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.wemportal entity: sensor.ww0_warmwassertemperatur
2023-03-10 10:46:17.188 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.wemportal entity: sensor.we0_leistung
2023-03-10 10:46:17.189 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.wemportal entity: sensor.we0_vorlauftemperatur
2023-03-10 10:46:17.189 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new sensor.wemportal entity: sensor.we0_anlagendruck
2023-03-10 10:46:17.226 INFO (MainThread) [homeassistant.components.select] Setting up select.wemportal
2023-03-10 10:46:17.231 INFO (MainThread) [homeassistant.components.number] Setting up number.wemportal
2023-03-10 10:46:17.288 INFO (MainThread) [homeassistant.components.switch] Setting up switch.wemportal

[dm82m]

some of the entities are migrated but a lot of the old sensors will become restored:

[dm82m]

and with release version I get >80 sensors and with dev version there are only 29 sensors...

[erikkastelec]

Did you change mode to both inside Home Assistant configurations?

[dm82m]

no I did not ... sorry for that!

tested it now and only a few sensors restored:

Home Assistant Core
2023-03-10 11:00:25.944 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration wemportal which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-03-10 11:00:29.154 INFO (MainThread) [homeassistant.bootstrap] Setting up stage 2: {'image_upload', 'hardware', 'automation', 'input_boolean', 'lovelace', 'zone', 'sun', 'diagnostics', 'hacs', 'counter', 'device_automation', 'history', 'media_source', 'map', 'conversation', 'person', 'persistent_notification', 'default_config', 'my', 'file_upload', 'scene', 'config', 'system_health', 'search', 'energy', 'wemportal', 'timer', 'logbook', 'input_datetime', 'application_credentials', 'blueprint', 'input_text', 'tts', 'repairs', 'mobile_app', 'tag', 'auth', 'onboarding', 'trace', 'script', 'input_button', 'schedule', 'input_select', 'homeassistant_alerts', 'analytics', 'input_number'}
2023-03-10 11:00:30.821 INFO (MainThread) [homeassistant.setup] Setting up wemportal
2023-03-10 11:00:30.823 INFO (MainThread) [homeassistant.setup] Setup of domain wemportal took 0.0 seconds
2023-03-10 11:00:31.306 DEBUG (SyncWorker_3) [custom_components.wemportal] Fetching api device data
2023-03-10 11:00:31.559 DEBUG (SyncWorker_3) [custom_components.wemportal] Fetching api parameters data
2023-03-10 11:00:35.761 DEBUG (SyncWorker_3) [custom_components.wemportal] Fetching fresh api data
2023-03-10 11:00:47.820 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 16.986 seconds (success: True)
2023-03-10 11:00:47.820 INFO (MainThread) [custom_components.wemportal] Migrating entity names for wemportal
2023-03-10 11:00:47.825 WARNING (MainThread) [homeassistant.helpers.frame] Detected integration that uses deprecated `async_get_registry` to access entity registry, use async_get instead. Please report issue to the custom integration author for wemportal using this method at custom_components/wemportal/__init__.py, line 46: er = await entity_registry.async_get_registry(hass)
2023-03-10 11:00:47.829 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.wemportal
2023-03-10 11:00:47.847 INFO (MainThread) [homeassistant.components.select] Setting up select.wemportal
2023-03-10 11:00:47.849 INFO (MainThread) [homeassistant.components.number] Setting up number.wemportal
2023-03-10 11:00:47.862 INFO (MainThread) [homeassistant.components.switch] Setting up switch.wemportal
2023-03-10 11:00:51.525 INFO (MainThread) [custom_components.wemportal] Migrating entity names for wemportal because of config entry update
2023-03-10 11:00:51.661 DEBUG (SyncWorker_3) [custom_components.wemportal] Fetching api device data
2023-03-10 11:00:51.832 DEBUG (SyncWorker_3) [custom_components.wemportal] Fetching api parameters data
2023-03-10 11:01:17.929 DEBUG (SyncWorker_3) [custom_components.wemportal] Fetching fresh api data
2023-03-10 11:01:29.112 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 37.579 seconds (success: True)
2023-03-10 11:01:29.113 INFO (MainThread) [custom_components.wemportal] Migrating entity names for wemportal
2023-03-10 11:01:29.115 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.system_status). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:system-status).
2023-03-10 11:01:29.116 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.system_aussentemperatur_aktuell). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:system-aussentemperatur_aktuell).
2023-03-10 11:01:29.116 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.system_aussentemperatur_gedampft). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:system-aussentemperatur_gedämpft).
2023-03-10 11:01:29.116 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.system_aussentemperatur_gemischt). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:system-aussentemperatur_gemischt).
2023-03-10 11:01:29.116 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.system_aussentemperatur_rucksetzen). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:system-aussentemperatur_rücksetzen).
2023-03-10 11:01:29.117 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.system_warmeanforderung_heizung_aktuell). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:system-wärmeanforderung_heizung_aktuell).
2023-03-10 11:01:29.117 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.system_warmeanforderung_heizkreis_aktuell_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:system-wärmeanforderung_heizkreis_aktuell_hzk1).
2023-03-10 11:01:29.117 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.system_warmeanforderung_warmwasser_aktuell). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:system-wärmeanforderung_warmwasser_aktuell).
2023-03-10 11:01:29.118 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_betriebsphase_wtc_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-betriebsphase_wtc_we0).
2023-03-10 11:01:29.118 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_betriebsphase_brenner_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-betriebsphase_brenner_we0).
2023-03-10 11:01:29.118 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_sollleistung_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-sollleistung_aktuell_we0).
2023-03-10 11:01:29.119 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_istleistung_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-istleistung_aktuell_we0).
2023-03-10 11:01:29.119 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_vorlaufsolltemperatur_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-vorlaufsolltemperatur_aktuell_we0).
2023-03-10 11:01:29.119 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_vorlauftemperatur_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-vorlauftemperatur_aktuell_we0).
2023-03-10 11:01:29.119 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_vorlauftemperatur_vpt_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-vorlauftemperatur_vpt_aktuell_we0).
2023-03-10 11:01:29.120 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_rucklauftemperatur_vpt_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-rücklauftemperatur_vpt_aktuell_we0).
2023-03-10 11:01:29.125 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_abgastemperatur_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-abgastemperatur_aktuell_we0).
2023-03-10 11:01:29.126 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_tageswarmemenge_vortag_gesamt_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-tageswärmemenge_(vortag)_gesamt_we0).
2023-03-10 11:01:29.126 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_tageswarmemenge_vortag_heizbetrieb_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-tageswärmemenge_(vortag)_heizbetrieb_we0).
2023-03-10 11:01:29.126 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_tageswarmemenge_vortag_warmwasserbetrieb_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-tageswärmemenge_(vortag)_warmwasserbetrieb_we0).
2023-03-10 11:01:29.127 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_zahler_seit_rucksetzen_brennerstarts_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-zähler_seit_rücksetzen_brennerstarts_we0).
2023-03-10 11:01:29.127 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_zahler_seit_rucksetzen_betriebsstunden_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-zähler_seit_rücksetzen_betriebsstunden_we0).
2023-03-10 11:01:29.127 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_zahler_seit_rucksetzen_rucksetzen_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-zähler_seit_rücksetzen_rücksetzen_we0).
2023-03-10 11:01:29.127 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_gesamtzahler_brennerstarts_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-gesamtzähler_brennerstarts_we0).
2023-03-10 11:01:29.128 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_gesamtzahler_betriebsstunden_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-gesamtzähler_betriebsstunden_we0).
2023-03-10 11:01:29.128 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_pumpenleistung_pumpe_intern_sollleistung_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-pumpenleistung_pumpe_intern_sollleistung_we0).
2023-03-10 11:01:29.128 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_pumpenleistung_pumpe_intern_elektrische_leistung_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-pumpenleistung_pumpe_intern_elektrische_leistung_we0).
2023-03-10 11:01:29.129 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_pumpenleistung_pumpe_intern_betriebsart_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-pumpenleistung_pumpe_intern_betriebsart_we0).
2023-03-10 11:01:29.129 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_volumenstrom_vpt_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-volumenstrom_vpt_aktuell_we0).
2023-03-10 11:01:29.129 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_warmeleistung_vpt_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-wärmeleistung_vpt_aktuell_we0).
2023-03-10 11:01:29.129 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_anlagendruck_vpt_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-anlagendruck_vpt_aktuell_we0).
2023-03-10 11:01:29.130 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_ionisationssignal_scot_basiswert_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-ionisationssignal_scot-basiswert_aktuell_we0).
2023-03-10 11:01:29.130 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_ionisationssignal_sollwert_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-ionisationssignal_sollwert_aktuell_we0).
2023-03-10 11:01:29.141 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_ionisationssignal_scot_istwert_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-ionisationssignal_scot-istwert_aktuell_we0).
2023-03-10 11:01:29.141 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_ionisationssignal_start_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-ionisationssignal_start_aktuell_we0).
2023-03-10 11:01:29.141 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_gasventil_offset_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-gasventil_offset_aktuell_we0).
2023-03-10 11:01:29.142 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_zeit_bis_flammenbildung_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-zeit_bis_flammenbildung_aktuell_we0).
2023-03-10 11:01:29.142 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_gasventil_ansteuersignal_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-gasventil_ansteuersignal_aktuell_we0).
2023-03-10 11:01:29.143 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_gas_luft_verhaltnis_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-gas-luft-verhältnis_aktuell_we0).
2023-03-10 11:01:29.143 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_geblasedrehzahl_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-gebläsedrehzahl_aktuell_we0).
2023-03-10 11:01:29.143 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.wtc_g_geblase_ansteuersignal_aktuell_we0). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:wtc-g-gebläse-ansteuersignal_aktuell_we0).
2023-03-10 11:01:29.143 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_betriebsart_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-betriebsart_hzk1).
2023-03-10 11:01:29.144 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_status_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-status_hzk1).
2023-03-10 11:01:29.144 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_aussentemperatur_lokal_gemittelt_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-aussentemperatur_lokal_gemittelt_hzk1).
2023-03-10 11:01:29.144 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_aussentemperatur_lokal_gemischt_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-aussentemperatur_lokal_gemischt_hzk1).
2023-03-10 11:01:29.144 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_aussentemperatur_aktuell_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-aussentemperatur_aktuell_hzk1).
2023-03-10 11:01:29.145 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_raumsolltemperatur_aktuell_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-raumsolltemperatur_aktuell_hzk1).
2023-03-10 11:01:29.145 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_vorlaufsolltemperatur_aktuell_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-vorlaufsolltemperatur_aktuell_hzk1).
2023-03-10 11:01:29.148 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_vorlauftemperatur_aktuell_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-vorlauftemperatur_aktuell_hzk1).
2023-03-10 11:01:29.148 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_mischerstellung_soll_aktuell_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-mischerstellung_soll_aktuell_hzk1).
2023-03-10 11:01:29.148 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_mischerstellung_ist_aktuell_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-mischerstellung_ist_aktuell_hzk1).
2023-03-10 11:01:29.149 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_pumpe_heizkreis_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-pumpe_heizkreis_hzk1).
2023-03-10 11:01:29.149 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_korrektur_aufheizoptimierung_letzte_optimierung_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-korrektur_aufheizoptimierung_letzte_optimierung_hzk1).
2023-03-10 11:01:29.149 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_korrektur_aufheizoptimierung_korrektur_aktuell_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-korrektur_aufheizoptimierung_korrektur_aktuell_hzk1).
2023-03-10 11:01:29.149 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_1_korrektur_aufheizoptimierung_rucksetzen_hzk1). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_1-korrektur_aufheizoptimierung_rücksetzen_hzk1).
2023-03-10 11:01:29.150 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_2_status). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_2-status).
2023-03-10 11:01:29.150 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_2_warmwassersolltemperatur_aktuell). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_2-warmwassersolltemperatur_aktuell).
2023-03-10 11:01:29.150 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_2_warmwassertemperatur_aktuell). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_2-warmwassertemperatur_aktuell).
2023-03-10 11:01:29.151 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heizkreis_heizkreise_heizkreis_2_pumpe_warmwasser). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heizkreis_heizkreise_heizkreis_2-pumpe_warmwasser).
2023-03-10 11:01:29.151 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.room_setpoint_temperature). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:room_setpoint_temperature).
2023-03-10 11:01:29.151 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.room_temperature). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:room_temperature).
2023-03-10 11:01:29.151 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.outside_temperature). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:outside_temperature).
2023-03-10 11:01:29.152 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.hot_water_temperature). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:hot_water_temperature).
2023-03-10 11:01:29.152 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.heat_output). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:heat_output).
2023-03-10 11:01:29.152 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.flow_temperature). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:flow_temperature).
2023-03-10 11:01:29.152 INFO (MainThread) [custom_components.wemportal] Found entity with old id (sensor.system_pressure). Updating to new unique_id (7f566e1ac6a0f60ef755b2b9cb85a830:4813:system_pressure).
2023-03-10 11:01:29.154 DEBUG (SyncWorker_0) [custom_components.wemportal] Fetching fresh api data
2023-03-10 11:01:40.237 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 11.084 seconds (success: True)
2023-03-10 11:01:40.238 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.wemportal
2023-03-10 11:01:40.238 INFO (MainThread) [homeassistant.components.number] Setting up number.wemportal
2023-03-10 11:01:40.238 INFO (MainThread) [homeassistant.components.select] Setting up select.wemportal
2023-03-10 11:01:40.239 INFO (MainThread) [homeassistant.components.switch] Setting up switch.wemportal

[erikkastelec]

I believe these sensors were created when the new integration was first set up with mode=api. When the mode changes, the names of some entities may also change. For now, I will leave them as is for now.

Since these are newly created sensors, we won't lose any old historical data. It's just a little inconvenient to delete them.

[dm82m]

okay, moved my production instance to dev branch.

it would be great if the options would be directly changeable on initial setup. currently I manually need to change them later on. and there is a typo:

[erikkastelec]

I would prefer if people read the documentation. If we put this information in the initial setup, many people may choose scan intervals that are too small and then wonder why the integration stops working after only a few hours. However, since this is a one-time setup, I don't think it should bother anyone too much.

[dm82m]

you should also switch the async_get to remove the deprecation warning: https://github.com/dm82m/hass-Deltasol-KM2/commit/480fdf38cb8b2d266fffcb833e0cd43ebf75cd23

so far again thank you for all the work! :)

[erikkastelec]

Done, thanks for the reminder.

[dm82m]

got a few error logs:

2023-03-10 14:23:11.165 DEBUG (SyncWorker_9) [custom_components.wemportal] Fetching fresh api data
2023-03-10 14:23:23.772 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 12.608 seconds (success: True)
2023-03-10 14:28:23.164 DEBUG (SyncWorker_9) [custom_components.wemportal] Fetching fresh api data
2023-03-10 14:28:35.453 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 12.289 seconds (success: True)
2023-03-10 14:33:35.166 DEBUG (SyncWorker_8) [custom_components.wemportal] Fetching fresh api data
2023-03-10 14:33:52.346 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 17.181 seconds (success: True)
2023-03-10 14:38:52.165 DEBUG (SyncWorker_2) [custom_components.wemportal] Fetching fresh api data
2023-03-10 14:39:10.314 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 18.150 seconds (success: True)
2023-03-10 14:44:11.645 DEBUG (SyncWorker_2) [custom_components.wemportal] Fetching fresh api data
2023-03-10 14:44:24.477 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 12.833 seconds (success: True)
2023-03-10 14:49:26.900 ERROR (MainThread) [custom_components.wemportal] Error fetching data from wemportal
File "/config/custom_components/wemportal/wemportalapi.py", line 116, in fetch_webscraping_data
data = processor.run([wemportal_job])[0]
File "/config/custom_components/wemportal/wemportalapi.py", line 98, in fetch_data
File "/config/custom_components/wemportal/wemportalapi.py", line 118, in fetch_webscraping_data
raise WemPortalError(
custom_components.wemportal.exceptions.WemPortalError: There was a problem with getting data from WEM Portal. If this problem persists, open an issue at https://github.com/erikkastelec/hass-WEM-Portal/issues
File "/config/custom_components/wemportal/coordinator.py", line 33, in _async_update_data
File "/config/custom_components/wemportal/wemportalapi.py", line 107, in fetch_data
raise WemPortalError from exc
custom_components.wemportal.exceptions.WemPortalError
2023-03-10 14:49:26.903 ERROR (MainThread) [custom_components.wemportal] Error fetching WemPortal update data:
2023-03-10 14:49:26.908 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 2.744 seconds (success: False)

restarted the integration and back working. will observe it.

[dm82m]

Error reoccurres:

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: custom_components.wemportal Source: custom_components/wemportal/wemportalapi.py:107 Integration: Weishaupt WEM Portal (documentation, issues) First occurred: 14:49:26 (30 occurrences) Last logged: 23:27:33

Error fetching data from wemportal Traceback (most recent call last): File "/config/custom_components/wemportal/wemportalapi.py", line 116, in fetch_webscraping_data data = processor.run([wemportal_job])[0] IndexError: list index out of range

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/config/custom_components/wemportal/wemportalapi.py", line 98, in fetch_data webscrapint_data = self.fetch_webscraping_data() File "/config/custom_components/wemportal/wemportalapi.py", line 118, in fetch_webscraping_data raise WemPortalError( custom_components.wemportal.exceptions.WemPortalError: There was a problem with getting data from WEM Portal. If this problem persists, open an issue at https://github.com/erikkastelec/hass-WEM-Portal/issues

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/config/custom_components/wemportal/coordinator.py", line 33, in _async_update_data return await self.hass.async_add_executor_job(self.api.fetch_data) File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/config/custom_components/wemportal/wemportalapi.py", line 107, in fetch_data raise WemPortalError from exc custom_components.wemportal.exceptions.WemPortalError

[dm82m]

All sensors are offline. After reloading the integration they are back for a few hour till this error occurs.

[erikkastelec]

I have ecountered the same issue with my instance, will look into it in the morning.

[erikkastelec]

Hopefully this solves the problem. I am testing this as we speak.

[dm82m]

will update to it and keep you posted; with 9707cf9 my cpu usage also increased to ~80% yesterday evening. will also keep an eye on that.

[dm82m]

worked for about for about 5 hours and then a exception but no autorecovery so far:

2023-03-11 16:52:17.895 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 16.585 seconds (success: True)
2023-03-11 16:57:20.665 ERROR (MainThread) [custom_components.wemportal] Error fetching data from wemportal
File "/config/custom_components/wemportal/wemportalapi.py", line 145, in fetch_webscraping_data
data = processor.run([wemportal_job])[0]
File "/config/custom_components/wemportal/wemportalapi.py", line 114, in fetch_data
File "/config/custom_components/wemportal/wemportalapi.py", line 151, in fetch_webscraping_data
raise WemPortalError(DATA_GATHERING_ERROR) from exc
custom_components.wemportal.exceptions.WemPortalError: An error occurred while gathering data. This issue should resolve by itself. If this problem persists, open an issue at https://github.com/erikkastelec/hass-WEM-Portal/issues
File "/config/custom_components/wemportal/coordinator.py", line 33, in _async_update_data
File "/config/custom_components/wemportal/wemportalapi.py", line 136, in fetch_data
raise WemPortalError from exc
custom_components.wemportal.exceptions.WemPortalError
2023-03-11 16:57:20.670 ERROR (MainThread) [custom_components.wemportal] Error fetching WemPortal update data:
2023-03-11 16:57:20.673 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 3.362 seconds (success: False)
2023-03-11 17:02:20.310 DEBUG (SyncWorker_0) [custom_components.wemportal] Fetching fresh api data
2023-03-11 17:02:20.515 ERROR (MainThread) [custom_components.wemportal] Error fetching data from wemportal
File "/config/custom_components/wemportal/wemportalapi.py", line 217, in make_api_call
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://www.wemportal.com/app/DataAccess/Refresh
File "/config/custom_components/wemportal/wemportalapi.py", line 187, in api_login
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://www.wemportal.com/app/Account/Login
File "/config/custom_components/wemportal/wemportalapi.py", line 129, in fetch_data
File "/config/custom_components/wemportal/wemportalapi.py", line 366, in get_data
File "/config/custom_components/wemportal/wemportalapi.py", line 220, in make_api_call
File "/config/custom_components/wemportal/wemportalapi.py", line 195, in api_login
raise ServerError("WemPortal server error") from exc
custom_components.wemportal.exceptions.ServerError: WemPortal server error
File "/config/custom_components/wemportal/coordinator.py", line 33, in _async_update_data
File "/config/custom_components/wemportal/wemportalapi.py", line 136, in fetch_data
raise WemPortalError from exc
custom_components.wemportal.exceptions.WemPortalError
2023-03-11 17:02:20.525 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 0.215 seconds (success: False)
2023-03-11 17:07:20.417 ERROR (MainThread) [custom_components.wemportal] Error fetching data from wemportal
File "/config/custom_components/wemportal/wemportalapi.py", line 187, in api_login
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://www.wemportal.com/app/Account/Login
File "/config/custom_components/wemportal/wemportalapi.py", line 87, in fetch_data
File "/config/custom_components/wemportal/wemportalapi.py", line 195, in api_login
raise ServerError("WemPortal server error") from exc
custom_components.wemportal.exceptions.ServerError: WemPortal server error
File "/config/custom_components/wemportal/coordinator.py", line 33, in _async_update_data
File "/config/custom_components/wemportal/wemportalapi.py", line 136, in fetch_data
raise WemPortalError from exc
custom_components.wemportal.exceptions.WemPortalError
2023-03-11 17:07:20.429 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 0.119 seconds (success: False)

[erikkastelec]

Same for me. I think the problem is in the spider session expiring. Will update you when I find something

[erikkastelec]

Should be fixed now. Will report back if it fails again.

[dm82m]

Updated. Will keep you posted.

[erikkastelec]

Problem persists for me, but the cause changed. Looking into it

[dm82m]

2023-03-12 02:16:55.387 DEBUG (SyncWorker_3) [custom_components.wemportal] Fetching fresh api data
2023-03-12 02:17:06.918 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 11.532 seconds (success: True)
2023-03-12 02:22:09.168 ERROR (SyncWorker_0) [custom_components.wemportal] An error occurred while gathering data.This issue should resolve by itself. If this problem persists,open an issue at https://github.com/erikkastelec/hass-WEM-Portal/issues
2023-03-12 02:22:09.174 DEBUG (SyncWorker_0) [custom_components.wemportal] Fetching fresh api data
2023-03-12 02:22:21.333 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 14.946 seconds (success: True)
2023-03-12 02:27:21.387 DEBUG (SyncWorker_1) [custom_components.wemportal] Fetching fresh api data
2023-03-12 02:27:33.033 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 11.647 seconds (success: True)
2023-03-12 02:32:33.386 DEBUG (SyncWorker_2) [custom_components.wemportal] Fetching fresh api data
2023-03-12 02:32:46.337 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 12.951 seconds (success: True)
2023-03-12 02:38:01.851 DEBUG (SyncWorker_10) [custom_components.wemportal] Fetching fresh api data
2023-03-12 02:38:14.126 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 27.737 seconds (success: True)
2023-03-12 02:43:14.387 DEBUG (SyncWorker_0) [custom_components.wemportal] Fetching fresh api data
2023-03-12 02:43:26.559 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 12.173 seconds (success: True)
2023-03-12 02:48:26.387 DEBUG (SyncWorker_10) [custom_components.wemportal] Fetching fresh api data
2023-03-12 02:48:37.776 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 11.390 seconds (success: True)
2023-03-12 02:53:37.386 DEBUG (SyncWorker_1) [custom_components.wemportal] Fetching fresh api data
2023-03-12 02:53:51.222 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 13.836 seconds (success: True)
2023-03-12 02:58:51.388 DEBUG (SyncWorker_2) [custom_components.wemportal] Fetching fresh api data
2023-03-12 02:59:03.742 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 12.355 seconds (success: True)
2023-03-12 03:04:03.385 DEBUG (SyncWorker_6) [custom_components.wemportal] Fetching fresh api data
2023-03-12 03:04:15.396 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 12.012 seconds (success: True)
2023-03-12 03:09:35.474 DEBUG (SyncWorker_6) [custom_components.wemportal] Fetching fresh api data
2023-03-12 03:09:47.463 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 32.074 seconds (success: True)
2023-03-12 03:14:47.386 DEBUG (SyncWorker_4) [custom_components.wemportal] Fetching fresh api data
2023-03-12 03:15:00.748 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 13.362 seconds (success: True)
2023-03-12 03:20:00.387 DEBUG (SyncWorker_8) [custom_components.wemportal] Fetching fresh api data
2023-03-12 03:20:15.780 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 15.394 seconds (success: True)
2023-03-12 03:25:15.386 DEBUG (SyncWorker_7) [custom_components.wemportal] Fetching fresh api data
2023-03-12 03:25:23.454 ERROR (SyncWorker_7) [custom_components.wemportal] WemPortal server error
2023-03-12 03:25:23.461 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 8.076 seconds (success: True)
2023-03-12 03:30:23.490 ERROR (MainThread) [custom_components.wemportal] Error fetching data from wemportal
File "/config/custom_components/wemportal/wemportalapi.py", line 194, in api_login
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://www.wemportal.com/app/Account/Login
File "/config/custom_components/wemportal/wemportalapi.py", line 87, in fetch_data
File "/config/custom_components/wemportal/wemportalapi.py", line 202, in api_login
raise ServerError("WemPortal server error") from exc
custom_components.wemportal.exceptions.ServerError: WemPortal server error
File "/config/custom_components/wemportal/coordinator.py", line 33, in _async_update_data
File "/config/custom_components/wemportal/wemportalapi.py", line 143, in fetch_data
raise WemPortalError from exc
custom_components.wemportal.exceptions.WemPortalError
2023-03-12 03:30:23.494 ERROR (MainThread) [custom_components.wemportal] Error fetching WemPortal update data:
2023-03-12 03:30:23.503 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 0.117 seconds (success: False)
2023-03-12 03:35:23.481 ERROR (MainThread) [custom_components.wemportal] Error fetching data from wemportal
File "/config/custom_components/wemportal/wemportalapi.py", line 194, in api_login
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://www.wemportal.com/app/Account/Login
File "/config/custom_components/wemportal/wemportalapi.py", line 87, in fetch_data
File "/config/custom_components/wemportal/wemportalapi.py", line 202, in api_login
raise ServerError("WemPortal server error") from exc
custom_components.wemportal.exceptions.ServerError: WemPortal server error
File "/config/custom_components/wemportal/coordinator.py", line 33, in _async_update_data
File "/config/custom_components/wemportal/wemportalapi.py", line 143, in fetch_data
raise WemPortalError from exc
custom_components.wemportal.exceptions.WemPortalError
2023-03-12 03:35:23.488 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 0.102 seconds (success: False)
2023-03-12 03:40:23.493 ERROR (MainThread) [custom_components.wemportal] Error fetching data from wemportal
File "/config/custom_components/wemportal/wemportalapi.py", line 194, in api_login
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://www.wemportal.com/app/Account/Login
File "/config/custom_components/wemportal/wemportalapi.py", line 87, in fetch_data
File "/config/custom_components/wemportal/wemportalapi.py", line 202, in api_login
raise ServerError("WemPortal server error") from exc
custom_components.wemportal.exceptions.ServerError: WemPortal server error
File "/config/custom_components/wemportal/coordinator.py", line 33, in _async_update_data
File "/config/custom_components/wemportal/wemportalapi.py", line 143, in fetch_data
raise WemPortalError from exc
custom_components.wemportal.exceptions.WemPortalError
2023-03-12 03:40:23.497 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 0.113 seconds (success: False)

[erikkastelec]

I added some additional logging to figure out the underlying issue. I wonder what happens at 3:30 AM, as my instance also failed at the same time.

Yesterdays error was caused by improper handling of spider session, but this seems to be a problem with API. It is quite weird to me that we get 500 server error, but maybe they provide some additional info inside response status and message, which are now logged.

[dm82m]

I already updated to it. Will observe and report.

[dm82m]

today in the morning after a server restart:

2023-03-13 08:17:11.226 WARNING (SyncWorker_3) [homeassistant.loader] We found a custom integration wemportal which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-03-13 08:17:26.844 INFO (MainThread) [homeassistant.bootstrap] Setting up stage 2: {'waste_collection_schedule', 'sonos', 'tts', 'zone', 'tapo_control', 'cover', 'analytics', 'input_number', 'scheduler', 'history', 'logbook', 'input_datetime', 'hacs', 'elgato', 'input_button', 'media_source', 'notify', 'image_upload', 'ios', 'light', 'cast', 'trace', 'schedule', 'config', 'device_automation', 'hardware', 'esphome', 'fritz', 'sun', 'wemportal', 'counter', 'zha', 'google_assistant', 'energy', 'conversation', 'default_config', 'group', 'ipp', 'automation', 'file_upload', 'lovelace', 'diagnostics', 'system_health', 'xiaomi_miio', 'webrtc', 'media_player', 'person', 'homematicip_local', 'repairs', 'onboarding', 'easee', 'thread', 'persistent_notification', 'utility_meter', 'tag', 'application_credentials', 'mobile_app', 'sensor', 'met', 'ffmpeg', 'timer', 'input_boolean', 'map', 'input_select', 'camera', 'search', 'scene', 'adguard', 'auth', 'homeassistant_alerts', 'script', 'my', 'device_tracker', 'input_text', 'python_script', 'nuki_ng', 'blueprint'}
2023-03-13 08:17:30.193 INFO (MainThread) [homeassistant.setup] Setting up wemportal
2023-03-13 08:17:30.194 INFO (MainThread) [homeassistant.setup] Setup of domain wemportal took 0.0 seconds
2023-03-13 08:17:30.467 DEBUG (SyncWorker_2) [custom_components.wemportal] Fetching api device data
2023-03-13 08:17:30.858 DEBUG (SyncWorker_2) [custom_components.wemportal] Fetching api parameters data
2023-03-13 08:17:58.249 DEBUG (SyncWorker_2) [custom_components.wemportal] Fetching fresh api data
2023-03-13 08:18:10.293 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 39.998 seconds (success: True)
2023-03-13 08:18:10.293 INFO (MainThread) [custom_components.wemportal] Migrating entity names for wemportal
2023-03-13 08:18:10.301 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.wemportal
2023-03-13 08:18:10.305 INFO (MainThread) [homeassistant.components.number] Setting up number.wemportal
2023-03-13 08:18:10.309 INFO (MainThread) [homeassistant.components.select] Setting up select.wemportal
2023-03-13 08:18:10.312 INFO (MainThread) [homeassistant.components.switch] Setting up switch.wemportal
2023-03-13 08:23:10.490 DEBUG (SyncWorker_5) [custom_components.wemportal] Fetching fresh api data
2023-03-13 08:23:21.670 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 11.181 seconds (success: True)
2023-03-13 08:28:21.490 DEBUG (SyncWorker_1) [custom_components.wemportal] Fetching fresh api data
2023-03-13 08:28:21.676 ERROR (SyncWorker_1) [custom_components.wemportal] WemPortal server error: Server returned internal status code: and message:
2023-03-13 08:28:21.683 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 0.194 seconds (success: True)
2023-03-13 08:33:21.588 ERROR (MainThread) [custom_components.wemportal] Error fetching data from wemportal
File "/config/custom_components/wemportal/wemportalapi.py", line 194, in api_login
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://www.wemportal.com/app/Account/Login
File "/config/custom_components/wemportal/wemportalapi.py", line 87, in fetch_data
File "/config/custom_components/wemportal/wemportalapi.py", line 203, in api_login
custom_components.wemportal.exceptions.ServerError: WemPortal server error: Server returned internal status code: and message:
File "/config/custom_components/wemportal/coordinator.py", line 33, in _async_update_data
File "/config/custom_components/wemportal/wemportalapi.py", line 143, in fetch_data
raise WemPortalError from exc
custom_components.wemportal.exceptions.WemPortalError
2023-03-13 08:33:21.600 ERROR (MainThread) [custom_components.wemportal] Error fetching WemPortal update data:
2023-03-13 08:33:21.603 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 0.115 seconds (success: False)

[erikkastelec]

My instance also failed at the same time. Will have to look further into this. Logging also does not seem to give any additional info.

If all else fails I will implement automatic reload of integration on server errors, which seems to fix this.

[dm82m]

I already thought about an small automation that reloads integration if sensor gets unavailable. But better if integration takes care on its own.

[erikkastelec]

Added api reload on update failure in dev branch.

[dm82m]

api reload also not helping:

2023-03-14 13:17:07.457 DEBUG (SyncWorker_2) [custom_components.wemportal] Fetching fresh api data
2023-03-14 13:17:19.837 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 12.381 seconds (success: True)
2023-03-14 13:22:19.459 DEBUG (SyncWorker_6) [custom_components.wemportal] Fetching fresh api data
2023-03-14 13:22:31.398 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 11.940 seconds (success: True)
2023-03-14 13:27:31.472 DEBUG (SyncWorker_9) [custom_components.wemportal] Fetching fresh api data
2023-03-14 13:27:31.730 ERROR (SyncWorker_9) [custom_components.wemportal] WemPortal server error: Server returned internal status code: and message:
2023-03-14 13:27:31.738 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 0.277 seconds (success: True)
2023-03-14 13:32:31.623 ERROR (MainThread) [custom_components.wemportal] Error fetching data from wemportal
File "/config/custom_components/wemportal/wemportalapi.py", line 194, in api_login
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://www.wemportal.com/app/Account/Login
File "/config/custom_components/wemportal/wemportalapi.py", line 87, in fetch_data
File "/config/custom_components/wemportal/wemportalapi.py", line 203, in api_login
custom_components.wemportal.exceptions.ServerError: WemPortal server error: Server returned internal status code: and message:
File "/config/custom_components/wemportal/coordinator.py", line 42, in _async_update_data
File "/config/custom_components/wemportal/wemportalapi.py", line 143, in fetch_data
raise WemPortalError from exc
custom_components.wemportal.exceptions.WemPortalError
2023-03-14 13:32:31.644 ERROR (MainThread) [custom_components.wemportal] Creating new wemportal api instance
2023-03-14 13:32:31.647 ERROR (MainThread) [custom_components.wemportal] Error fetching WemPortal update data:
2023-03-14 13:32:31.649 DEBUG (MainThread) [custom_components.wemportal] Finished fetching WemPortal update data in 0.188 seconds (success: False)
2023-03-14 13:37:31.620 ERROR (MainThread) [custom_components.wemportal] Error fetching data from wemportal
File "/config/custom_components/wemportal/wemportalapi.py", line 194, in api_login
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://www.wemportal.com/app/Account/Login
File "/config/custom_components/wemportal/wemportalapi.py", line 87, in fetch_data
File "/config/custom_components/wemportal/wemportalapi.py", line 203, in api_login
custom_components.wemportal.exceptions.ServerError: WemPortal server error: Server returned internal status code: and message:
File "/config/custom_components/wemportal/coordinator.py", line 42, in _async_update_data
File "/config/custom_components/wemportal/wemportalapi.py", line 143, in fetch_data
raise WemPortalError from exc
custom_components.wemportal.exceptions.WemPortalError

[erikkastelec]

I will look if I can trigger a reload of the whole component from inside the component (currently I am only creating a new API object). I can't really understand why that would be needed, but I guess it is.

Expect next failure at 18:30 (they occur every 29 h, regardless of when component was started)

[erikkastelec]

This update should fix the api not being replaced with a new instance on failure

[dm82m]

Looks good so far. The only thing I got was this:

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: custom_components.wemportal
Source: custom_components/wemportal/sensor.py:138 
Integration: Weishaupt WEM Portal (documentation, issues) 
First occurred: 19. März 2023 um 13:12:23 (13 occurrences) 
Last logged: 20. März 2023 um 21:21:31

Can't find f7ad0827d125d6220474a66849c330ae:4813:system_pressure
Can't find f7ad0827d125d6220474a66849c330ae:4813:heizkreis_heizkreise_heizkreis_2-status
Can't find f7ad0827d125d6220474a66849c330ae:4813:heizkreis_heizkreise_heizkreis_2-warmwassertemperatur_aktuell

[dm82m]

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: custom_components.wemportal Source: custom_components/wemportal/wemportalapi.py:136 Integration: Weishaupt WEM Portal (documentation, issues) First occurred: 18:12:33 (29 occurrences) Last logged: 20:39:44

'NoneType' object has no attribute 'status_code'

[dm82m]

Also a Integration restart is not helping:

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: homeassistant.config_entries Source: custom_components/wemportal/coordinator.py:42 Integration: Weishaupt WEM Portal (documentation, issues) First occurred: 20:43:20 (1 occurrences) Last logged: 20:43:20

Error setting up entry yyy@xxx-online.de for wemportal Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 383, in async_setup result = await component.async_setup_entry(hass, self) File "/config/custom_components/wemportal/init.py", line 104, in async_setup_entry await coordinator.async_config_entry_first_refresh() File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 204, in async_config_entry_first_refresh await self._async_refresh( File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 239, in _async_refresh self.data = await self._async_update_data() File "/config/custom_components/wemportal/coordinator.py", line 42, in _async_update_data return await self.hass.async_add_executor_job(self.api.fetch_data) asyncio.exceptions.CancelledError

[erikkastelec]

This should fix the issue. I think that this was also the reason behind "Can't find ..." errors you reported previously.

[dm82m]

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: homeassistant Source: custom_components/wemportal/select.py:121 Integration: Weishaupt WEM Portal (documentation, issues) First occurred: 02:54:25 (57 occurrences) Last logged: 07:36:26

Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 182, in _handle_refresh_interval await self._async_refresh(log_failures=True, scheduled=True) File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 330, in _async_refresh self.async_update_listeners() File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 135, in async_update_listeners update_callback() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 559, in async_write_ha_state self._async_write_ha_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 600, in _async_write_ha_state state = self._stringify_state(available) File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 565, in _stringify_state if (state := self.state) is None: File "/usr/src/homeassistant/homeassistant/components/select/init.py", line 147, in state if self.current_option is None or self.current_option not in self.options: File "/config/custom_components/wemportal/select.py", line 121, in current_option self.coordinator.data[self._device_id][self._name]["value"] KeyError: 'HZK1-U_Raumsoll'