DWD cant fetch data anymore (v2.1.8)

[panhans]

First thank your for your hard work! With the latest version the component isn't working for my anymore.

Version of home_assistant

2024.9.1

Version of the custom_component

2.1.8

(I have to downgrade to 2.1.7)

Describe the bug

No data is provided anymore.

Debug log

2024-09-16 13:05:15.341 ERROR (MainThread) [custom_components.dwd_weather] Unexpected error fetching DWD Weather Coordinator for 10162 data
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 354, in _async_refresh
self.data = await self._async_update_data()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 271, in _async_update_data
return await self.update_method()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/dwd_weather/connector.py", line 97, in async_update
if await self._hass.async_add_executor_job(self._update):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/dwd_weather/connector.py", line 106, in _update
self.dwd_weather.update(
File "/usr/local/lib/python3.12/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 673, in update
self.download_latest_kml(self.station_id, force_hourly)
File "/usr/local/lib/python3.12/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 1013, in download_latest_kml
self.parse_kml(kml)
File "/usr/local/lib/python3.12/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 706, in parse_kml
self.loaded_station_name = self.parse_station_name(tree)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 758, in parse_station_name
return tree.xpath(
^^^^^^^^^^^
IndexError: list index out of range
2024-09-16 13:05:15.342 DEBUG (MainThread) [custom_components.dwd_weather] Finished fetching DWD Weather Coordinator for 10162 data in 11.769 seconds (success: False)
2024-09-16 13:05:15.342 DEBUG (MainThread) [custom_components.dwd_weather] issue_time: 2024-09-16 10:00:00+00:00
2024-09-16 13:05:15.342 DEBUG (MainThread) [custom_components.dwd_weather] ConfigEntryNotReady
2024-09-16 13:05:20.854 ERROR (MainThread) [homeassistant.core] Error running job: <Job onetime listen homeassistant_start <function _async_at_core_state.<locals>._matched_event at 0x7fc1afb75120> HassJobType.Callback <_OneTimeListener homeassistant.helpers.start:<function _async_at_core_state.<locals>._matched_event at 0x7fc1afb75120>>>
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 657, in state
numerical_value = int(value)
^^^^^^^^^^
ValueError: invalid literal for int() with base 10: ''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 660, in state
numerical_value = float(value)
^^^^^^^^^^^^
ValueError: could not convert string to float: ''
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/core.py", line 1562, in async_fire_internal
self._hass.async_run_hass_job(job, event)
File "/usr/src/homeassistant/homeassistant/core.py", line 937, in async_run_hass_job
hassjob.target(*args)
File "/usr/src/homeassistant/homeassistant/core.py", line 1423, in __call__
self.hass.async_run_hass_job(self.listener_job, event)
File "/usr/src/homeassistant/homeassistant/core.py", line 937, in async_run_hass_job
hassjob.target(*args)
File "/usr/src/homeassistant/homeassistant/helpers/start.py", line 44, in _matched_event
hass.async_run_hass_job(at_start_job, hass)
File "/usr/src/homeassistant/homeassistant/core.py", line 937, in async_run_hass_job
hassjob.target(*args)
File "/usr/src/homeassistant/homeassistant/components/template/template_entity.py", line 487, in _async_template_startup
result_info.async_refresh()
File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 1122, in async_refresh
self._refresh(None)
File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 1308, in _refresh
self.hass.async_run_hass_job(self._job, event, updates)
File "/usr/src/homeassistant/homeassistant/core.py", line 937, in async_run_hass_job
hassjob.target(*args)
File "/usr/src/homeassistant/homeassistant/components/template/template_entity.py", line 436, in _handle_results
self.async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1005, in async_write_ha_state
self._async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1130, in _async_write_ha_state
self.__async_calculate_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1067, in __async_calculate_state
state = self._stringify_state(available)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1011, in _stringify_state
if (state := self.state) is None:

[FL550]

Hi, thanks for reporting!

Do you habe the hourly update enabled?

[panhans]

Yes, after disabling it, it works again. So there is a problem with that feature, right?

[FL550]

Yes, i have updated the hourly download to resolve an issue with large memory consumption. This however might have introduced this error.

There is at least one other user who have reported this. I'll have a look into this soon.

[lk3de]

I can confirm this issue (DWD 2.1.8, HA 2024.9.1). After disabling the hourly update and reloading the integration, the issue is resolved instantly.

Logs:

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: custom_components.dwd_weather
Quelle: helpers/update_coordinator.py:354
Integration: Deutscher Wetterdienst (Dokumentation, Probleme)
Erstmals aufgetreten: 10:56:16 (10 Vorkommnisse)
Zuletzt protokolliert: 11:01:15

Unexpected error fetching DWD Weather Coordinator for Q440 data
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 354, in _async_refresh
    self.data = await self._async_update_data()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 271, in _async_update_data
    return await self.update_method()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dwd_weather/connector.py", line 97, in async_update
    if await self._hass.async_add_executor_job(self._update):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dwd_weather/connector.py", line 106, in _update
    self.dwd_weather.update(
  File "/usr/local/lib/python3.12/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 673, in update
    self.download_latest_kml(self.station_id, force_hourly)
  File "/usr/local/lib/python3.12/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 1013, in download_latest_kml
    self.parse_kml(kml)
  File "/usr/local/lib/python3.12/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 706, in parse_kml
    self.loaded_station_name = self.parse_station_name(tree)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 758, in parse_station_name
    return tree.xpath(
           ^^^^^^^^^^^
IndexError: list index out of range

[FL550]

I implemented a change, which should resolve this error. Please report if the error still exists.

[panhans]

I'd just updated to the latest version. Everything works as expected. Thanks!

[panhans]

Sry, the fix caused a new bug.

Here is my weather data without hourly refresh (correct):

And here with hourly refresh:

[FL550]

Hm, i'll check it again.

[FL550]

I've found the error and fixed it with the latest version :)

[klatka]

On v2.1.11 this bug seems to be back

[FL550]

Can you please provide more info on this. Best would be the log output and your station I'd.

[klatka]

Logger: custom_components.dwd_weather
Quelle: helpers/update_coordinator.py:354
Integration: Deutscher Wetterdienst (Dokumentation, Probleme)
Erstmals aufgetreten: 10:25:26 (15 Vorkommnisse)
Zuletzt protokolliert: 10:36:34

Unexpected error fetching DWD Weather Coordinator for H432 data
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 354, in _async_refresh
    self.data = await self._async_update_data()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 271, in _async_update_data
    return await self.update_method()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dwd_weather/connector.py", line 97, in async_update
    if await self._hass.async_add_executor_job(self._update):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dwd_weather/connector.py", line 106, in _update
    self.dwd_weather.update(
  File "/usr/local/lib/python3.12/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 674, in update
    self.download_latest_kml(self.station_id, force_hourly)
  File "/usr/local/lib/python3.12/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 1028, in download_latest_kml
    self.parse_kml(kml)
  File "/usr/local/lib/python3.12/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 707, in parse_kml
    self.loaded_station_name = self.parse_station_name(tree)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 760, in parse_station_name
    return tree.xpath(
           ^^^^^^^^^^^
IndexError: list index out of range

[tco99ttocs]

got the same prob. with the 2.1.11. I switched back to 2.1.10 and the problem is still there. If i disable the hourly update its working! The "funny" thing is, Version 2.1.11 worked fine until today in the evening!!

[FL550]

I've updated the integration. Hopefully it's resolved now.

[klatka]

Resolved. Thanks!