Weatherflow API seems to have broken the integration

[robchandhok]

See also #80 I think.

Home Assistant 2023.8.4 Supervisor 2023.08.3 Operating System 10.5 Frontend 20230802.1 - latest

I just stopped being able to read my Tempest. When you reload, the log reports:

2023-08-28 12:49:19.004 WARNING (MainThread) [homeassistant.config_entries] Config entry 'TheView' for weatherflow integration not ready yet: Error while retreiving forecast data: Error occured processing forecast data. Error message: 'sea_level_pressure'; Retrying in background

I turned on debugging and see this pattern:

2023-08-28 12:50:38.459 DEBUG (MainThread) [custom_components.weatherflow] Finished fetching weatherflow data in 0.218 seconds (success: True)
2023-08-28 12:50:38.878 DEBUG (MainThread) [custom_components.weatherflow] Finished fetching weatherflow data in 0.419 seconds (success: False)
2023-08-28 12:51:59.649 DEBUG (MainThread) [custom_components.weatherflow] Finished fetching weatherflow data in 0.195 seconds (success: True)
2023-08-28 12:52:00.141 DEBUG (MainThread) [custom_components.weatherflow] Finished fetching weatherflow data in 0.492 seconds (success: False)

My guess is that something changed in the API return values?

I am quite puzzled. My parents instance of HA (not up to 2023.8.4) still works with their tempest.

I have tried:

• deleting and re-adding the integration
• redownloading via HACS
• making a new access token and using it
• accessing the API via their URLs and my token, and that worked

[eelcovb]

I have exactly the same versions / issue:

in my log:

2023-08-28 16:27:52.329 DEBUG (MainThread) [custom_components.weatherflow] Finished fetching weatherflow data in 0.102 seconds (success: True) 2023-08-28 16:27:52.642 DEBUG (MainThread) [custom_components.weatherflow] Finished fetching weatherflow data in 0.313 seconds (success: False) 2023-08-28 16:27:52.642 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Orlando-TerraceNET' for weatherflow integration not ready yet: Error while retreiving forecast data: Error occured processing forecast data. Error message: 'sea_level_pressure'; Retrying in background

Tried the exact same things, rebooted everything multiple times.

[jeremyhamm]

My integration is also broken with WeatherFlow

Logs: Logger: homeassistant.config_entries Source: config_entries.py:430 First occurred: 2:51:24 PM (1 occurrences) Last logged: 2:51:24 PM

Config entry '**' for weatherflow integration not ready yet: Error while retreiving forecast data: Error occured processing forecast data. Error message: 'sea_level_pressure'; Retrying in background

[jeremyhamm]

@robchandhok Any idea what version of HA your Parents are running. I reverted to an older backup (2023.8.3) and it was still broken.

[jgosnell56]

I am on 2023.8.4 of HA and seeing this same error. HA log shows: 2023-08-28 16:42:50.552 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Stratford' for weatherflow integration not ready yet: Error while retreiving forecast data: Error occured processing forecast data. Error message: 'sea_level_pressure'; Retrying in background

API from my weather station still reports sea_level_pressure so not quite sure why we are suddenly seeing this error. Excerpt from the API webpage of my weather station: { "elevation": 189.06375122070312, "is_public": true, "latitude": xxxxx, "longitude": xxxxx, "obs": [ { "air_density": 1.15988, "air_temperature": 23.8, "barometric_pressure": 988.7, "brightness": 5308, "delta_t": 0.5, "dew_point": 23.1, "feels_like": 23.8, "heat_index": 23.8, "lightning_strike_count": 1, "lightning_strike_count_last_1hr": 178, "lightning_strike_count_last_3hr": 223, "lightning_strike_last_distance": 10, "lightning_strike_last_epoch": 1693264013, "precip": 0.807931, "precip_accum_last_1hr": 14.031167, "precip_accum_local_day": 19.700306, "precip_accum_local_day_final": 20.045038, "precip_accum_local_yesterday": 0, "precip_accum_local_yesterday_final": 0, "precip_analysis_type_yesterday": 0, "precip_minutes_local_day": 90, "precip_minutes_local_yesterday": 0, "precip_minutes_local_yesterday_final": 0, "pressure_trend": "rising", "relative_humidity": 96, "sea_level_pressure": 1011.4, "solar_radiation": 44, "station_pressure": 988.7, "timestamp": 1693264013, "uv": 0.47, "wet_bulb_globe_temperature": 23.7, "wet_bulb_temperature": 23.3, "wind_avg": 0.2, "wind_chill": 23.8, "wind_direction": 332, "wind_gust": 0.7, "wind_lull": 0 }

[jeremyhamm]

@jgosnell56 Not sure if it matters to you but you may want to hide your exact location.

[ragd011]

I can confirm same issue started earlier today. As luck would have it, we lost power at 3:40 ET and when it came back, this integration is no longer working. Completely updated system.

Logger: homeassistant.config_entries Source: config_entries.py:1290 First occurred: 8:39:58 PM (1 occurrences) Last logged: 8:39:58 PM

Config entry 'The House' for weatherflow integration not ready yet: Error while retreiving forecast data: Error occured processing forecast data. Error message: 'sea_level_pressure'; Retrying in background

Mine started work today 8/29 at 10am without doing anything???

[eelcovb]

I had exactly the same power outage "luck".. Checked my whole setup 6 times, changed the DB, threw away existing sensor data, just to find a fix..

[gitBrave-hub]

it is broken for me too, as described above by robchandhok

[idaband]

well time to dig into this code....uggg fun times LOL. I already commented out some stuff...with no luck.

[idaband]

well time to dig into this code....uggg fun times LOL. I already commented out some stuff...with no luck.

so looking at the straight up forcast api data that comes back... i don't see an issue with that json.... which means...something is getting messed up as its reads the data back is my guess, which could be the python library maybe?

  "current_conditions": {
    "air_density": 1.2,
    "air_temperature": 13,
    "brightness": 0,
    "conditions": "Clear",
    "delta_t": 1,
    "dew_point": 12,
    "feels_like": 13,
    "icon": "clear-night",
    "is_precip_local_day_rain_check": true,
    "is_precip_local_yesterday_rain_check": true,
    "lightning_strike_count_last_1hr": 0,
    "lightning_strike_count_last_3hr": 0,
    "lightning_strike_last_distance": 41,
    "lightning_strike_last_distance_msg": "39 - 43 km",
    "lightning_strike_last_epoch": 1692936462,
    "precip_accum_local_day": 0,
    "precip_accum_local_yesterday": 0,
    "precip_minutes_local_day": 0,
    "precip_minutes_local_yesterday": 0,
    "pressure_trend": "rising",
    "relative_humidity": 93,
    "sea_level_pressure": 1016.6,
    "solar_radiation": 0,
    "station_pressure": 983,
    "time": 1693275901,
    "uv": 0,
    "wet_bulb_globe_temperature": 16,
    "wet_bulb_temperature": 12,
    "wind_avg": 0,
    "wind_direction": 0,
    "wind_direction_cardinal": "N",
    "wind_gust": 0
  },

[robchandhok]

@robchandhok Any idea what version of HA your Parents are running. I reverted to an older backup (2023.8.3) and it was still broken.

2023.8.3

I am now convinced it's a bug in the config/init flow. So restart breaks it. Running instances are fine. Can't see it though.

[robchandhok]

Annnnnndddd….. it’s back working. There still might be an opportunity to catch this at init (defensive programming) but I don’t hav3 an example to test against. Hopefully working for others.

[ZeddicPrime]

Just reloaded the plugin and it's still broken for me.

[robchandhok]

Hmmm. Race condition?

[Yomegaman]

My Weatherflow died today too. I've tried deleting & reinstalling (HA Supervisor) but get these error lines;

ERROR:asyncio:Task exception was never retrieved future: <Task finished name='Task-7' coro=<WeatherFlowMqtt._update_forecast() done, defined at /usr/local/lib/python3.10/site-packages/weatherflow2mqtt-3.1.6-py3.10.egg/weatherflow2mqtt/weatherflow_mqtt.py:755> exception=KeyError('sea_level_pressure')> Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/weatherflow2mqtt-3.1.6-py3.10.egg/weatherflow2mqtt/weatherflow_mqtt.py", line 760, in _update_forecast if any(forecast := await self.forecast.update_forecast()): File "/usr/local/lib/python3.10/site-packages/weatherflow2mqtt-3.1.6-py3.10.egg/weatherflow2mqtt/forecast.py", line 187, in update_forecast row["sea_level_pressure"] KeyError: 'sea_level_pressure'

Then it sets up all my sensors with lines like this; INFO:weatherflow2mqtt.weatherflow_mqtt:Setting up Tempest sensor: Status

Finally giving these errors; Traceback (most recent call last): File "/usr/local/bin/weatherflow2mqtt", line 33, in sys.exit(load_entry_point('weatherflow2mqtt==3.1.6', 'console_scripts', 'weatherflow2mqtt')()) File "/usr/local/lib/python3.10/site-packages/weatherflow2mqtt-3.1.6-py3.10.egg/weatherflow2mqtt/main.py", line 10, in main File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/usr/local/lib/python3.10/site-packages/weatherflow2mqtt-3.1.6-py3.10.egg/weatherflow2mqtt/weatherflow_mqtt.py", line 859, in main File "/usr/local/lib/python3.10/site-packages/weatherflow2mqtt-3.1.6-py3.10.egg/weatherflow2mqtt/weatherflow_mqtt.py", line 232, in run_time_based_updates File "/usr/local/lib/python3.10/site-packages/weatherflow2mqtt-3.1.6-py3.10.egg/weatherflow2mqtt/weatherflow_mqtt.py", line 760, in _update_forecast File "/usr/local/lib/python3.10/site-packages/weatherflow2mqtt-3.1.6-py3.10.egg/weatherflow2mqtt/forecast.py", line 187, in update_forecast KeyError: 'sea_level_pressure'

This has worked great for about a year - until today.

PLEASE HELP

[UpDryTwist]

I also started encountering this after a Home Assistant restart yesterday (using the cloud-based integration, not the local MQTT integration). Like the others, if I call the API myself, it appears to return well-formed JSON, with the appropriate "sea_level_pressure" key. I can't see any difference between that key and surrounding keys. I walked through the code (inspection, not debugger), and couldn't see anything anomalous in the code -- nothing that indicates why "sea_level_pressure" would hiccough and not any other value (if, say, it was indeed a race condition, I wouldn't expect us all to have the exact same experience). I think someone's going to have to look at this in a debugger (or with a lot of debugging statements turned on) to catch it. (If no one has a chance to do so, I may have some time this coming weekend, but certainly not before then.)

One hypothesis: different JSON is returned for some reason when we're all invoking it in the browser than when it's being invoked from within Home Assistant (although this doesn't really make sense, as it seems to happen both for folks using the cloud-based API and the local MQTT-based API). First thing to do, I think, would be to see if it reproduces when just calling https://github.com/briis/pyweatherflowrest from a command-line harness. Just a thought.

[idaband]

I haven't checked weatherflow community forum yet either... They might have info of changes

Get Outlook for Androidhttps://aka.ms/AAb9ysg

---

From: UpDryTwist @.> Sent: Tuesday, August 29, 2023 9:01:09 AM To: briis/hass-weatherflow @.> Cc: Green Eggs and Code @.>; Comment @.> Subject: Re: [briis/hass-weatherflow] Weatherflow API seems to have broken the integration (Issue #81)

I also started encountering this after a Home Assistant restart yesterday (using the cloud-based integration, not the local MQTT integration). Like the others, if I call the API myself, it appears to return well-formed JSON, with the appropriate "sea_level_pressure" key. I can't see any difference between that key and surrounding keys. I walked through the code (inspection, not debugger), and couldn't see anything anomalous in the code -- nothing that indicates why "sea_level_pressure" would hiccough and not any other value (if, say, it was indeed a race condition, I wouldn't expect us all to have the exact same experience). I think someone's going to have to look at this in a debugger (or with a lot of debugging statements turned on) to catch it. (If no one has a chance to do so, I may have some time this coming weekend, but certainly not before then.)

One hypothesis: different JSON is returned for some reason when we're all invoking it in the browser than when it's being invoked from within Home Assistant (although this doesn't really make sense, as it seems to happen both for folks using the cloud-based API and the local MQTT-based API). First thing to do, I think, would be to see if it reproduces when just calling https://github.com/briis/pyweatherflowrest from a command-line harness. Just a thought.

— Reply to this email directly, view it on GitHubhttps://github.com/briis/hass-weatherflow/issues/81#issuecomment-1697401660, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADRHLMD7KOW7ZFUQ7I54FPLXXXRZLANCNFSM6AAAAAA4B52ZAQ. You are receiving this because you commented.Message ID: @.***>

[robchandhok]

Agree with @UpDryTwist . It's starting to look like the forecast API call is the culprit, it would be nice to still succeed with the station data in the face of that error (regardless of this specific error many of us are experiencing). I'm not a good enough python programmer to make that change tho. Just needs a wrap in an error catch.

[JtwoA]

@briis - any feedback?

[briis]

I have been away for a long time, so I did not monitor this for a while. I will have a look at it ASAP.

[id628]

FYI, this might help. The other Home Assistant integration is having the same issue: https://github.com/briis/hass-weatherflow2mqtt/issues/234

A stated workaround there is:

Until there is a fix, a workaround is to remove STATION_ID and STATION_TOKEN from the config page for the weatherflow2mqtt add-on (this prevents the add-on from trying to get the forecast)

(I have not verified that this helps, since I'm running this one...)

(EDIT: I didn't realize both projects are by the same author! Cool!!!)

[eelcovb]

Haven't looked into it thoroughly, but my 2 cents:

Looking at a piece of forecast code, I noticed 'weather_daily' and 'weather_hourly' are used as keys:

`_WEATHER_DAILY = "weather_daily" _WEATHER_HOURLY = "weather_hourly"

_LOGGER = logging.getLogger(name)

WEATHER_TYPES: tuple[WeatherEntityDescription, ...] = ( WeatherEntityDescription( key=_WEATHER_DAILY, name="Day based Forecast", ), WeatherEntityDescription( key=_WEATHER_HOURLY, name="Hourly based Forecast", ), )`

However, on checking the API response, the forecast keys used seem to be 'daily' and 'hourly' (https://apidocs.tempestwx.com/reference/get_better-forecast). Again, only took a quick look, will try to look into it and test hopefully later today. We have a hurricane coming up, and it'd be great to get everything recorded in HA.

Cheers

[JtwoA]

We have a hurricane coming up, and it'd be great to get everything recorded in HA. @eelcovb Where are you in the path? Lowcountry of SC here.

[eelcovb]

We have a hurricane coming up, and it'd be great to get everything recorded in HA. @eelcovb Where are you in the path? Lowcountry of SC here.

Orlando, FL no direct exposure, but probably some tornado's. Stay safe!

[eelcovb]

Haven't looked into it thoroughly, but my 2 cents:

Looking at a piece of forecast code, I noticed 'weather_daily' and 'weather_hourly' are used as keys:

`_WEATHER_DAILY = "weather_daily" _WEATHER_HOURLY = "weather_hourly"

_LOGGER = logging.getLogger(name)

WEATHER_TYPES: tuple[WeatherEntityDescription, ...] = ( WeatherEntityDescription( key=_WEATHER_DAILY, name="Day based Forecast", ), WeatherEntityDescription( key=_WEATHER_HOURLY, name="Hourly based Forecast", ), )`

However, on checking the API response, the forecast keys used seem to be 'daily' and 'hourly' (https://apidocs.tempestwx.com/reference/get_better-forecast). Again, only took a quick look, will try to look into it and test hopefully later today. We have a hurricane coming up, and it'd be great to get everything recorded in HA.

Cheers

Checked it, not the issue.

[ldeffenb]

East coast of Florida and my HA-integrated Tempest has also gone offline. Talk about (bad) timing!

[UpDryTwist]

OK, thought I wouldn't have time until this weekend, but it was bugging me (and it's driving me nuts looking at my dashboard with errors on it!), so I pulled it up in the debugger. @briis if you didn't already figure this out, the problem is that the hourly forecast no longer includes sea_level_pressure (I wonder if there was just a firmware push?). I'll confirm a fix and put up a pull request this evening.

For others on this thread, after I confirm, I'll let you know how you might patch until Bjarne has a chance to cut a release.

EDIT: Yup. Confirmed the fix. Will put up the pull request later, but for you hurricane trackers, below are general guidelines for how you can fix this in your installation. WARNING: There are many ways to run Home Assistant. I cannot speak for your own instance (nor, unfortunately, do I have the bandwidth today to troubleshoot other instances). You need to be reasonably technical to apply this patch; you need to understand the mechanics of your Home Assistance instance; and you need to be comfortable patching Python. You could break things!

• The error occurs in pyweatherflowrest/api.py. The Weatherflow API has changed (firmware update?) in the last day or two to no longer return sea_level_pressure and uv in the hourly forecasts. This results in a KeyError exception in api.py
• I have patched this by providing a more resilient way of reading the hourly forecast values (as maybe this was a defect on the Weatherflow side, and they'll be restored). You can find this fix in my fork at: https://github.com/UpDryTwist/pyweatherflowrest/blob/main/pyweatherflowrest/api.py
• This patch adds logging (so that future changes here will be easier to diagnose). By default, my end version will error out, unless you edit the file to set a value (IGNORE_FETCH_ERRORS = True). This is a bit clunky, but I think that @briis is better-positioned to figure out a cleaner way to support a parameter in the two different implementations that call this.
• You can find a version with this already set to True for you (so you don't need to edit) here: https://github.com/UpDryTwist/pyweatherflowrest/blob/b167b56b914bd24db155787928e05a7edf452773/pyweatherflowrest/api.py

Here are the steps I took to patch my version:

• Go to the link above for the file (confirm that IGNORE_FETCH_ERRORS = True at the top). Click "Raw" and grab the URL.
• Here's where your instructions may vary. You need to find where pyweatherflowrest is installed for your Home Assistant instance. There could be lots of places, which will vary by how you run, which OS you're using, etc., so I can't answer this for you. For me, I run in a container. So I opened a bash shell in my container and went to /usr/local/lib/python3.11/site-packages/pyweatherflowrest.
• From there, I renamed the old file (mv api.py api.py-save)
• Then I downloaded the patched file (wget <>)
• . . . and restarted Home Assistant.

Voila:

[UpDryTwist]

OK, @briis , the PR is there. Thanks, BTW, for this fantastic implementation! You can see how freaked out we all get when it's not working!

https://github.com/briis/pyweatherflowrest/pull/3

[idaband]

I can confirm the change works. I was able to make the change inside HA. I'm using the OS version.

[eelcovb]

OK, @briis , the PR is there. Thanks, BTW, for this fantastic implementation! You can see how freaked out we all get when it's not working!

briis/pyweatherflowrest#3

Works perfectly! thanks

[briis]

I have now release 1.0.10 of pyweatherflowrest which contains the PR done by @UpDryTwist . If everything goes well I will integrate this in a new release of this Integration later today.

[JtwoA]

If everything goes well I will integrate this in a new release of this Integration later today.

If you get it out by 18:00 EDT that would be just ahead of the storm hitting my location :-)

[idaband]

I have now release 1.0.10 of pyweatherflowrest which contains the PR done by @UpDryTwist . If everything goes well I will integrate this in a new release of this Integration later today.

How does HA pick up there was a depedancy change? Does the HACs also have to update in order to make HA pull an update with updated depedancy?

[PYehl73]

Mine doesn't have an api.py file to replace? my files are in folder just called weatherflowrest - not pyweatherflowrest?

[briis]

Once I publish a new release HACS will catch that automatically. If you can't wait for HACS to find, you can go to HACS, click the Integration and then on that page, click the 3 Dots in the Upper right Corner, and then Update Information. This will check immediatly.

I will release it now, with only the fix for this Issue, and then start looking at the other open Issues.

[briis]

1.0.15 is now released and should show up in HACS in a little while. I will close this now, please reopen if the problem persist.

[janick]

I just upgraded to 1.0.15 via HACS and the same error is still there.

[eelcovb]

Same here. I had it running with the manual fix. That was overridden by the update and now it's broken again.

[chheiss]

Same here, not working.

[UpDryTwist]

Mine doesn't have an api.py file to replace? my files are in folder just called weatherflowrest - not pyweatherflowrest?

@PYehl73, you need to find where the Python package pyweatherflowrest is installed in your system (it's a package install, not the custom component itself, which just uses this package), and that's where you'll find api.py, if you're trying to patch right away. Waiting for the fixed overall version to be released will be your best bet if that sentence doesn't make sense to you or you're not comfortable patching Python packages.

[jeremyhamm]

Updated the integration to 1.0.15 and it's still broken.

[Colorado4Wheeler]

Same here, 1.0.15 did not fix the issue.

[robchandhok]

Also remember the original issue is intermittent. For example, mine is working again and I haven't installed the new version. So there may also be something timing related between asynch calls?

[Colorado4Wheeler]

@robchandhok that's interesting, I haven't had it come up once since it went down, according to my history I can see that once it became unavailable it never returned.

[ldeffenb]

Mine went out at 10:07am Eastern yesterday morning, was back from 3:05pm to 5:36pm, then down until 5:19 this morning and it's been up and working since.

I'm wondering if they have a round-robin DNS and only some of the servers have the issue and other servers are still working as expected?

[Colorado4Wheeler]

Perhaps. This did force me to create a "Smart Weather" template where it can use whichever of my weather services are available in order of my preference so I always have a "master" weather device - because I have so much riding on the values from Weatherflow and they all broke, now it can fall back to OpenWeatherMap, Accuweather or NWS. I never needed it before but now I'm redefining everything that leaned on WeatherFlow to use the template system so there is never any down time.

But, as of now, and after several reboots and reloads and integration restarts - no WeatherFlow.

[chheiss]

I have two stations, the primary is working, the secondary is out since yesterday.

[chheiss]

I have two stations, the primary is working, the secondary is out since yesterday. V 15 did not makeit better.

[robchandhok]

Weatherflow appear to have at least 4 round robin servers for the api.

@briis maybe re-open this or tell us where to give you feedback? And thank you VERY much for the integration!

[briis]

I simply can't reproduce your issues here. I had a test instance running now for 8 hours - 1 time it couldn't fetch the data, but it worked on the next scheduled call.

Please give me some more logs from Home Assistant so that I can see where this is failing.

Try also to read the raw data from your station directly from here: https://weatherflow.github.io/Tempest/api/swagger/