Irradiance forecasts no longer available

[almiho]

Hi there,

I have recently upgraded to HA 2023.9.3 and I have upgraded dwd_weather through the GUI to the latest version (should be 2.0.1).

I have been using my own template sensors to calculate total rest of day or rolling 24 hour irradiance forecasts and the like based on dwd_weather to have an idea how much heat to expect from my solar panels in the near future and to make decisions off of that.

Here is one example that used to work for several months prior to the two upgrades (HA and dwd_weather) I made:

- platform: template
  sensors:    
    sun_irradiance_staaken_rest_of_day_forecast:
      friendly_name: "Rest of day forecast of irradiance"
      unit_of_measurement: 'W/m^2' 
      value_template: >-
        {% set irrTotal = namespace(value=0) %}
        {% set current_hour = now().hour %}
        {% set hours_left = 24 - current_hour %}
        {% for irr in state_attr('sensor.sun_irradiance_staaken', 'data') -%}
          {% if loop.index <= hours_left %}
            {% set irrTotal.value = irrTotal.value + irr.value %}
          {% else %}
            {% break %}
          {% endif %}
        {%- endfor %}
        {{ irrTotal.value }}

After the upgrades I found the following error in the logs:

TemplateError('TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'') while processing template 'Template<template=({% set irrTotal = namespace(value=0) %} {% set current_hour = now().hour %} {% set hours_left = 24 - current_hour %} {% for irr in state_attr('sensor.sun_irradiance_staaken', 'data') -%} {% if loop.index <= hours_left %} {% set irrTotal.value = irrTotal.value + irr.value %} {% else %} {% break %} {% endif %} {%- endfor %} {{ irrTotal.value }}) renders=4>' for attribute '_attr_native_value' in entity 'sensor.sun_irradiance_staaken_rest_of_day_forecast'

I checked the sensor sensor.sun_irradiance_staaken in the developer tools and it showed the sensor's state as "unknown" and the attribute values looked like this:

state_class: measurement data:

• datetime: '2023-10-04T19:00:00.000Z' value: null
• datetime: '2023-10-04T20:00:00.000Z' value: null
• datetime: '2023-10-04T21:00:00.000Z' value: null
• datetime: '2023-10-04T22:00:00.000Z' value: null
• datetime: '2023-10-04T23:00:00.000Z' value: null (...)

Before the upgrades I would see numeric values instead of the null values

Since I head read the notes on the new trigger/action in templates and the breaking change in dwd_weather in the latest release I thought it might have to do with that and I updated my sensor definition to:

- template:
  - trigger:
      - platform: time_pattern
        hours: "/1"
      - platform: homeassistant
        event: start
      - platform: event
        event_type: event_template_reloaded
    action:
      - service: weather.get_forecast
        data:
          type: hourly
        target:
          entity_id: weather.staaken_staaken
        response_variable: irradiance_data
    sensor:
      - name: "Rest of day forecast of irradiance"
        unique_id: sun_irradiance_staaken_rest_of_day_forecast
        state: >-
          {% set irrTotal = namespace(value=0) %}
          {% set current_hour = now().hour %}
          {% set hours_left = 24 - current_hour %}
          {% for irr in irradiance_data.data -%}
            {% if loop.index <= hours_left %}
              {% set irrTotal.value = irrTotal.value + irr.value %}
            {% else %}
              {% break %}
            {% endif %}
          {%- endfor %}
          {{ irrTotal.value }}
        unit_of_measurement: "W/m^2" 

after that the error changed and it now is:

Template variable warning: 'dict object' has no attribute 'data' when rendering '{% set irrTotal = namespace(value=0) %} {% set current_hour = now().hour %} {% set hours_left = 24 - current_hour %} {% for irr in irradiance_data.data -%} {% if loop.index <= hours_left %} {% set irrTotal.value = irrTotal.value + irr.value %} {% else %} {% break %} {% endif %} {%- endfor %} {{ irrTotal.value }}'

so I manually tested the new get_forecast service like:

service: weather.get_forecast
data:
  type: hourly
target:
  entity_id: weather.staaken_staaken

and I got this:

forecast:

• datetime: "2023-10-04T19:00:00Z" condition: cloudy wind_bearing: SW wind_gusts: 5.14 precipitation_probability: 11 temperature: 14 templow: 14 wind_speed: 2.57 precipitation: 0 (...)

This tells me two things:

1. I will slightly have to adjust my code as the structure has changed a bit
2. there is no point in fixing the issue above as the attribute I have always been using (irradiance) is no longer part of the forecast

Obviously I am missing something but I can't figure out what it is. Any hints how to solve this and get the hourly forecasts for irradiance back would be highly appreciated!

Thanks, Alex

[diplix]

i have a similar issue (station „X104“). the sensors sensor.precipitation_probability_berlin_tegel and sensor.berlin_tegel_niederschlagsdauer (i renamed the first entity id, btw) are unavailable and i get this error:

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: homeassistant
Source: custom_components/dwd_weather/connector.py:215
Integration: Deutscher Wetterdienst (documentation, issues)
First occurred: 19:39:04 (10 occurrences)
Last logged: 19:48:07

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/update_coordinator.py", line 233, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/update_coordinator.py", line 389, in _async_refresh
    self.async_update_listeners()
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/update_coordinator.py", line 172, in async_update_listeners
    update_callback()
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
    self._async_write_ha_state()
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
    state, attr = self._async_generate_attributes()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/entity.py", line 785, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/home/homeassistant/.homeassistant/custom_components/dwd_weather/sensor.py", line 288, in state
    result = self._connector.get_precipitation_probability()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/.homeassistant/custom_components/dwd_weather/connector.py", line 289, in get_precipitation_probability
    return self.get_weather_value(WeatherDataType.PRECIPITATION_PROBABILITY)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/.homeassistant/custom_components/dwd_weather/connector.py", line 215, in get_weather_value
    value = self.dwd_weather.get_reported_weather(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 350, in get_reported_weather
    return self.report_data[weatherDataType.value[0]]
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'wwP'

the attribute values of both sensors are null, too.

so it seems that either the station does not offer those particular values or that theres a bug in the simple_dwd_weatherforecast library that occurs while processing these values.

either way, it would be better to handle the first case more gracefully, i.e. indicating that the sensor values are not available at this particular station by disabling the sensors. but hopefully it’s a fixable bug :)

[FL550]

@almiho:

1. I will slightly have to adjust my code as the structure has changed a bit there is no point in fixing the issue above as the attribute

There shouln't be changes to the sensor values. The problem is somewhere else.

2. I have always been using (irradiance) is no longer part of the forecast

The forecast breaking change is only related to the weather entity so you should be able to use them if we can get it working for you again.

Can you please update to the latest version and upload the debug log, so I can troubleshoot?

[FL550]

@diplix the station X104 you have mentioned should have the sensor values you are looking for. The error looks like some kind of network error, as this specific case should not happen. Can you file a new issue for this and upload your debug log as well? Thanks.

[almiho]

@almiho:

1. I will slightly have to adjust my code as the structure has changed a bit there is no point in fixing the issue above as the attribute

There shouln't be changes to the sensor values. The problem is somewhere else.

Got it, I think I was assuming that I would have to use the weather.get_forecast service rather than the ssensor sensor.sun_irradiance_staaken which I had used before. More than happy to keep things the same if we can make it work again.

2. I have always been using (irradiance) is no longer part of the forecast

The forecast breaking change is only related to the weather entity so you should be able to use them if we can get it working for you again.

The reason why I was assuming that it might have to do with the upgrade is that everything was working all the way up until I upgraded both HA and dwd_weather.

Can you please update to the latest version and upload the debug log, so I can troubleshoot?

Sure thing, here you go.

home-assistant_dwd_weather_2023-10-05T15-38-24.286Z.log

Thanks for your help! Alex

[diplix]

@diplix the station X104 you have mentioned should have the sensor values you are looking for. The error looks like some kind of network error, as this specific case should not happen. Can you file a new issue for this and upload your debug log as well? Thanks.

i looked at it more closly in #87 and there actually is no error but rather it’s expected behavior if you enable force_hourly like i did …

[FL550]

This is now fixed in v2.0.3. Please let me know if not and otherwise please close this issue.

[almiho]

I have upgraded HA 10.0 -> 10.1 and dwd_weather 2.0.2 -> 2.0.3 (and restarted HA, of course)

Unfortunately it looks like the symptoms are still the same for me:

is there anything else I should be doing?

[FL550]

Can you please give me the station id, so I can reproduce this error?

[almiho]

sure, it is Station N7651

this is really interesting, looks like pretty much everything but irradiance works

[FL550]

Unfortunately I have to confirm, that DWD has removed the data from this station :(

See here: https://opendata.dwd.de/weather/local_forecasts/mos/MOSMIX_L/single_stations/N7651/kml/MOSMIX_L_LATEST_N7651.kmz at the key Rad1h

[Failure404]

I have a similar problem with a particular station

`Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 233, in _handle_refresh_interval await self._async_refresh(log_failures=True, scheduled=True) File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 389, in _async_refresh self.async_update_listeners() File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 172, in async_update_listeners update_callback() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state self._async_write_ha_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state state, attr = self._async_generate_attributes() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes state = self._stringify_state(available) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state if (state := self.state) is None: ^^^^^^^^^^ File "/config/custom_components/dwd_weather/sensor.py", line 296, in state result = self._connector.get_sun_duration() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/dwd_weather/connector.py", line 306, in get_sun_duration return self.get_weather_value(WeatherDataType.SUN_DURATION) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/dwd_weather/connector.py", line 220, in get_weather_value value = self.dwd_weather.get_reported_weather( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 326, in get_reported_weather return self.report_data[weatherDataType.value[0]]

KeyError: 'SunD1'`

However, this key does exist in the dataset.
https://opendata.dwd.de/weather/local_forecasts/mos/MOSMIX_L/single_stations/H744/kml/MOSMIX_L_LATEST_H744.kmz

Overall, I am getting errors for precipitation duration, precipitation probability, sun duration and fog probability.

[almiho]

Unfortunately I have to confirm, that DWD has removed the data from this station :(

See here: https://opendata.dwd.de/weather/local_forecasts/mos/MOSMIX_L/single_stations/N7651/kml/MOSMIX_L_LATEST_N7651.kmz at the key Rad1h

That's almost funny, this means Murphy must have decided that the station in Staaken removes that exact attribute which I am using in that exact night in which I run my update on HA, well .. what should I say.

Anyways, I chose a different station and I got the sensors to work again but I still see error messages and exception from dwd_weather in the protocol.

I am still running HA 2023.10.1 and I have upgraded to dwd_weather 2.0.4 now.

I have removed the old weather station "Staaken" and I added a new one from FU Berlin

After starting up HA I see two error message in the protocol, the first one is

Logger: homeassistant.components.sensor Source: helpers/entity_platform.py:507 Integration: Sensor (documentation, issues) First occurred: 23:23:55 (1 occurrences) Last logged: 23:23:55

Error adding entities for domain sensor with platform dwd_weather Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 507, in async_add_entities await asyncio.gather(*tasks) File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 752, in _async_add_entity await entity.add_to_platform_finish() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1057, in add_to_platform_finish self.async_write_ha_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state self._async_write_ha_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state state, attr = self._async_generate_attributes() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes state = self._stringify_state(available) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state if (state := self.state) is None: ^^^^^^^^^^ File "/config/custom_components/dwd_weather/sensor.py", line 288, in state result = self._connector.get_precipitation_probability() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/dwd_weather/connector.py", line 294, in get_precipitation_probability return self.get_weather_value(WeatherDataType.PRECIPITATION_PROBABILITY) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/dwd_weather/connector.py", line 220, in get_weather_value value = self.dwd_weather.get_reported_weather( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 326, in get_reported_weather return self.report_data[weatherDataType.value[0]]

KeyError: 'wwP'

and the second one is:

Logger: homeassistant.components.sensor Source: helpers/entity_platform.py:367 Integration: Sensor (documentation, issues) First occurred: 23:23:55 (1 occurrences) Last logged: 23:23:55

Error while setting up dwd_weather platform for sensor Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 367, in _async_setup_platform await asyncio.gather(pending) File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 507, in async_add_entities await asyncio.gather(tasks) File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 752, in _async_add_entity await entity.add_to_platform_finish() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1057, in add_to_platform_finish self.async_write_ha_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state self._async_write_ha_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state state, attr = self._async_generate_attributes() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes state = self._stringify_state(available) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state if (state := self.state) is None: ^^^^^^^^^^ File "/config/custom_components/dwd_weather/sensor.py", line 288, in state result = self._connector.get_precipitation_probability() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/dwd_weather/connector.py", line 294, in get_precipitation_probability return self.get_weather_value(WeatherDataType.PRECIPITATION_PROBABILITY) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/dwd_weather/connector.py", line 220, in get_weather_value value = self.dwd_weather.get_reported_weather( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 326, in get_reported_weather return self.report_data[weatherDataType.value[0]]

KeyError: 'wwP'

if I wait for 3 minutes another error message shows up:

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: homeassistant Source: custom_components/dwd_weather/connector.py:220 Integration: Deutscher Wetterdienst (documentation, issues) First occurred: 23:24:54 (3 occurrences) Last logged: 23:26:54

Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 233, in _handle_refresh_interval await self._async_refresh(log_failures=True, scheduled=True) File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 389, in _async_refresh self.async_update_listeners() File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 172, in async_update_listeners update_callback() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state self._async_write_ha_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state state, attr = self._async_generate_attributes() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes state = self._stringify_state(available) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state if (state := self.state) is None: ^^^^^^^^^^ File "/config/custom_components/dwd_weather/sensor.py", line 288, in state result = self._connector.get_precipitation_probability() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/dwd_weather/connector.py", line 294, in get_precipitation_probability return self.get_weather_value(WeatherDataType.PRECIPITATION_PROBABILITY) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/dwd_weather/connector.py", line 220, in get_weather_value value = self.dwd_weather.get_reported_weather( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 326, in get_reported_weather return self.report_data[weatherDataType.value[0]]

KeyError: 'wwP'

any idea what causes these errors and how I could fix them?

Thanks, Alex

[fl0x53]

Hi all, I think I could trace down the problem. It is based on the "mixed" mode for data retrieval, that's the default mode you are presented in a pop-up when configuring the integration: "mixed_data": "Use report data if available and forecasted data if not (recommended)",

Thus, sensors are created that correspond to a certain selection of attributes of the MOSMIX_L KML files ("forecast_data"). When retrieving the "report_data", a different data source is used (CSV files), which provides only a smaller set of attributes. And here lies the problem in "simple_dwd_weatherforecast/dwdforecast.py". class WeatherDataType(Enum) defines e.g. PRECIPITATION_PROBABILITY = ["wwP", ""] # Unit: % (0..100), where wwP is the key of the attribute of the MOSMIX data, but there is simply no equivalent in the CSV data. Resulting in the KeyError: 'wwP' error as seen above.

For now, a workaround is to set up new stations and explicitly selecting report or forecast data.

[FL550]

Hi, thanks for helping troubleshooting.

I think I could trace down the problem. It is based on the "mixed" mode for data retrieval, that's the default mode you are presented in a pop-up when configuring the integration:

No, this is not the cause. It lies within the library simple_dwd_weatherforecast

class WeatherDataType(Enum) defines e.g. PRECIPITATION_PROBABILITY = ["wwP", ""] # Unit: % (0..100), where wwP is the key of the attribute of the MOSMIX data, but there is simply no equivalent in the CSV data. Resulting in the KeyError: 'wwP' error as seen above.

This is correct. I fixed the code in 2.0.5 so this error should not happen again.

[Failure404]

I just updated to 2.0.5, rebooted HA and removed & re-added the weather station, but still:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 233, in _handle_refresh_interval await self._async_refresh(log_failures=True, scheduled=True) File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 389, in _async_refresh self.async_update_listeners() File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 172, in async_update_listeners update_callback() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state self._async_write_ha_state() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state state, attr = self._async_generate_attributes() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes state = self._stringify_state(available) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state if (state := self.state) is None: ^^^^^^^^^^ File "/config/custom_components/dwd_weather/sensor.py", line 314, in state result = self._connector.get_precipitation_probability() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/dwd_weather/connector.py", line 294, in get_precipitation_probability return self.get_weather_value(WeatherDataType.PRECIPITATION_PROBABILITY) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/dwd_weather/connector.py", line 220, in get_weather_value value = self.dwd_weather.get_reported_weather( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 326, in get_reported_weather return self.report_data[weatherDataType.value[0]]

KeyError: 'wwP'

[FL550]

Can you please provide me the line with the configuration from the debug logs as well?

[fl0x53]

Hi, thanks for helping troubleshooting.

You're welcome

No, this is not the cause. It lies within the library simple_dwd_weatherforecast

Exactly, that's what I wrote one line below 😉

This is correct. I fixed the code in 2.0.5 so this error should not happen again.

Error still occurs. Attached debug log: home-assistant_dwd_weather_2023-10-16T14-26-34.331Z.log

I can reproduce the error on calling dwd_weather.get_reported_weather(dwdforecast.WeatherDataType.PRECIPITATION_PROBABILITY, True) directly from the library. That is what happens within the HA integration. Whereas the corresponding call for the forecast data works dwd_weather.get_forecast_data(dwdforecast.WeatherDataType.PRECIPITATION_PROBABILITY, time_now) I hope that helps somehow 🤷

[FL550]

I hope that helps somehow 🤷

Oh yes, indeed! I found the error. Thanks for helping! The next version should fix this.

[FL550]

New version is out

[Failure404]

Some new errors:

Logger: homeassistant.util.async_
Source: util/async_.py:137
First occurred: 17:00:14 (20 occurrences)
Last logged: 17:09:44
Detected blocking call to putrequest inside the event loop. This is causing stability issues. 
Please report issue to the custom integration author for dwd_weather 
doing blocking calls at custom_components/dwd_weather/connector.py, line 213: 
return markdownify(self.dwd_weather.get_weather_report(), strip=["br"])

and

Logger: homeassistant
Source: custom_components/dwd_weather/connector.py:213
Integration: Deutscher Wetterdienst
First occurred: 13:56:13 (357 occurrences)
Last logged: 16:59:43

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 233, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 389, in _async_refresh
    self.async_update_listeners()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 172, in async_update_listeners
    update_callback()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
    state, attr = self._async_generate_attributes()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/config/custom_components/dwd_weather/sensor.py", line 297, in state
    self._connector.get_weather_report(),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dwd_weather/connector.py", line 213, in get_weather_report
    return markdownify(self.dwd_weather.get_weather_report(), strip=["br"])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/markdownify/__init__.py", line 242, in markdownify
    return MarkdownConverter(**options).convert(html)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/markdownify/__init__.py", line 64, in convert
    soup = BeautifulSoup(html, 'html.parser')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/bs4/__init__.py", line 315, in __init__
    elif len(markup) <= 256 and (
         ^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()

and

Logger: homeassistant
Source: util/async_.py:151
First occurred: 17:00:14 (20 occurrences)
Last logged: 17:09:44
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 233, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 389, in _async_refresh
    self.async_update_listeners()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 172, in async_update_listeners
    update_callback()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
    state, attr = self._async_generate_attributes()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/config/custom_components/dwd_weather/sensor.py", line 297, in state
    self._connector.get_weather_report(),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dwd_weather/connector.py", line 213, in get_weather_report
    return markdownify(self.dwd_weather.get_weather_report(), strip=["br"])
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 764, in get_weather_report
    self.update(with_report=True)
  File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 549, in update
    self.download_latest_kml(self.station_id, force_hourly)
  File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 790, in download_latest_kml
    request = requests.get(url, headers=headers)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 714, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 415, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/lib/python3.11/http/client.py", line 1286, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1297, in _send_request
    self.putrequest(method, url, **skips)
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 219, in putrequest
    return _HTTPConnection.putrequest(self, method, url, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 164, in protected_loop_func
    check_loop(func, strict=strict)
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 151, in check_loop
    raise RuntimeError(
RuntimeError: Blocking calls must be done in the executor or a separate thread; Use `await hass.async_add_executor_job()`; at custom_components/dwd_weather/connector.py, line 213: return markdownify(self.dwd_weather.get_weather_report(), strip=["br"]

[almiho]

I have updated to 2.0.5 and I have switched again the weather station, this time to 10381: Berlin-Dahlem. I did stick to the recommended "data fetching" approach. The result is, everything seems to work fine for me now.

Thanks very much for your help and thanks very much developing this module in the first place!

As far as I am concerned the case can be closed. Should we keep it open because there is still a parallel discussion going on?

Thanks, Alex

Just in case someone is curios here is one of the sensors I did to control a few things subject to expected performance of my PV system. If you spot anything that isn't used as intended or that doesn't make sense. Again, as per my experience it works fine like this:

    berlin_irradiance_forecast_rest_of_day:
      friendly_name: "Berlin irradiance forecast rest of day"
      unit_of_measurement: 'W/m^2' 
      value_template: >-
        {% set irrTotal = namespace(value=0) %}
        {% set current_hour = now().hour %}
        {% set hours_left = 24 - current_hour %}
        {% set irradiance_data = states.sensor.berlin_dahlem_sonneneinstrahlung.attributes.data | default([]) %}
        {% if irradiance_data is not none %}
          {% for irr in irradiance_data %}
            {% if loop.index <= hours_left %}
              {% set irrTotal.value = irrTotal.value + irr.value %}
            {% else %}
              {% break %}
            {% endif %}
          {% endfor %}
        {{ irrTotal.value }}
        {% else %}
          unavailable
        {% endif %}

[FL550]

@almiho I'm glad to hear this!

Yes, leave this open for a moment longer until all issues mentioned here are resolved.

@Failure404 This error arises because you have enabled the weather report (the text one) without an available report. This is now fixed in the new version 2.0.7