KeyError in Pirate Weather Integration Causes Sensor Update Failure in Home Assistant

cloudbr34k84 commented 3 weeks ago

Describe the bug

The issue is occurring within the native_value function in the pirateweather/sensor.py file, specifically at line 1079. The error is due to a KeyError, which indicates that the key 'hrrr_subh' is missing in the data dictionary returned by Pirate Weather's API or the data object you're working with.

This error originated from a custom integration.

Logger: homeassistant
Source: custom_components/pirateweather/sensor.py:1079
integration: Pirate Weather (documentation, issues)
First occurred: 2:53:52 pm (1 occurrences)
Last logged: 2:53:52 pm

Error doing job: Task exception was never retrieved (None)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 1047, in _async_update_entity_states
    await asyncio.gather(*tasks)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 960, in async_update_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1130, in _async_write_ha_state
    self.__async_calculate_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1067, in __async_calculate_state
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1011, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 542, in state
    value = self.native_value
            ^^^^^^^^^^^^^^^^^
  File "/config/custom_components/pirateweather/sensor.py", line 1079, in native_value
    model_time_string = self._weather_coordinator.data.json["flags"][
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'hrrr_subh'

Error Explanation: KeyError: 'hrrr_subh': This means the code is trying to access the key 'hrrr_subh' inside the self._weather_coordinator.data.json["flags"] dictionary, but this key is not present in the API response or data. File "/config/custom_components/pirateweather/sensor.py", line 1079: The code at this line expects a key named 'hrrr_subh' to exist in the JSON data returned by Pirate Weather, but it doesn't.

Possible Causes: API Data Change: Pirate Weather's API might have changed, and the expected key 'hrrr_subh' is no longer part of the response. Incomplete Data: The API might not always return the 'hrrr_subh' key, depending on location, time, or other factors. Incorrect Data Handling: The integration might not be handling missing keys properly

Expected behavior

Successful Data Retrieval: The Pirate Weather integration should retrieve the necessary weather data without raising any errors, even if some optional keys (like 'hrrr_subh') are missing from the response.

Graceful Handling of Missing Data: If a specific key, such as 'hrrr_subh', is not present in the API response, the integration should handle it gracefully. This could involve using a default value, skipping the key, or logging a warning instead of raising an exception that causes the sensor to fail.

Seamless Sensor Update: The sensor should update its state using available data and display the correct weather information in Home Assistant. If some data is missing or unavailable, it should not cause the entire sensor to fail, but rather fallback to a safe behavior like using a placeholder or logging the issue without disruption.

Actual behavior

Error Raised Due to Missing Key: Instead of handling the missing 'hrrr_subh' key gracefully, the Pirate Weather integration raises a KeyError. This exception occurs because the code assumes that the key is always present in the API response, but it is not in this case.

Task Exception Never Retrieved: The error message "Task exception was never retrieved" indicates that the system encountered an unhandled exception during an asynchronous task, and it was not properly dealt with. As a result, the sensor fails to update as expected.

Sensor State Fails to Update: The sensor that relies on the Pirate Weather data cannot retrieve its state, leading to a failure in the sensor’s update process. This results in incomplete or missing weather information in Home Assistant.

Logs Display KeyError: The error is logged in the Home Assistant system logs, showing the specific line of code where the issue occurred (line 1079 of the pirateweather/sensor.py file), along with the KeyError for the 'hrrr_subh' key.

In summary, instead of retrieving and updating the sensor state with available data, the system encounters an error due to the missing 'hrrr_subh' key, which leads to the sensor not updating and logs an exception in Home Assistant.

Home Assistant version

2024.9.1

Integration version

1.5.8

Other details

No response

Troubleshooting steps

[X] I have updated my Home Assistant installation to the latest version.
[X] I have updated the Pirate Weather Integration to the latest version.
[X] I have gone through the documentation before opening this issue.
[X] I have searched this repository and API Repository to see if the issue has already been reported.
[X] I have restarted my Home Assistant installation.
[X] I have queried the API in my browser to confirm the issue is not with the API.
[X] I have written an informative title.

cloneofghosts commented 2 weeks ago

Sorry it looks like this issue got missed. Just to confirm you are still seeing this issue? If so then whereabouts are you approximately located? I'll also ping @alexander0042 to look into this to see if he has any solutions.

I've been running the HRRR Subhourly update sensor in my HA install and haven't seen any errors. When the data wasn't there the sensor was showing Unavailable and the sensors continued to update.

cloudbr34k84 commented 2 weeks ago

Yeah im seeing this issue im located Sunshine Coast, QLD Australia

cloneofghosts commented 2 weeks ago

Thanks for the information which confirmed my suspicion. The HRRR and NBM models are only available in North America. Where you're located in Australia the only models available are GFS and GEFS.

While not a permanent solution you can remove the NBM and HRRR update_time sensors and everything should start working again. As you mentioned in the issue there should be some sort of check to see if the data exists so it doesn't break the integration. If this issue is still open by the weekend I'll try and take a look to see if I can fix the issue.

cloneofghosts commented 1 week ago

Got this fixed and will release the update shortly. I thought about adding a warning but with the way things work it'll continuously spam the log with warnings which isn't ideal.

The sensors will show as Unknown instead of showing a time if there is no data available

Pirate-Weather / pirate-weather-ha