search

naofireblade / homebridge-weather-plus

A comprehensive weather plugin for homebridge.
MIT License
311 stars 61 forks source link

Fix tile labels, add sunrise/sunset and report sensor failure #285

Closed dacarson closed 3 months ago

dacarson commented 4 months ago

This fork has three fixes on it:

279 Report failing sensors. Shows as a general failure in app, also logs failure

247 Make use of ConfiguredName for compatibility values, to show values in the Home app where a native sensor isn't available

283 Report Sunset/Sunrise in forecast and current conditions when using Openweather API 2.5, and report Sunset/Sunrise for current conditions when using Openweather 3.0 API

Tested by running in my setup for a week, and 3.0 API was tested as documented here: https://github.com/jkreileder/homebridge-weather-plus/commit/e320cb91fd1499ea6773ce5d90d01edd63235696#r138496941

dacarson commented 4 months ago

I received the JSON config and I put it in my config. I am not seeing the error yet. Though to avoid the crash, and help debugging, I made this change to forecast fetching (https://github.com/naofireblade/homebridge-weather-plus/pull/285/commits/37409db2a05b04e2b283abad6b35533e36d64e5d). If it hits this issue again, it will log the response so that I can debug it 😊

wimleers commented 4 months ago

Thursday, just before midnight, looking at the forecast for tomorrow: IMG_7855

Friday, well past midnight: IMG_7858

No log entries at all. The crash from the other day hasn’t occurred since. I suspect it was an ephemeral server issue over at Weatherflow.

But this problem seems to lie in the plugin.

dacarson commented 4 months ago

This problem is because I only update once an hour. The previous forecast 23:59 and when you checked again at 00:30, an hour has not passed so the forecast has not updated. If you checked at 00:59 or 01:00 I believe it'll have a new forecast.

wimleers commented 4 months ago

This problem is because I only update once an hour. The previous forecast 23:59 and when you checked again at 00:30, an hour has not passed so the forecast has not updated. If you checked at 00:59 or 01:00 I believe it'll have a new forecast.

Good theory! 😄 But unfortunately the Eve app (restarted this morning just to make sure) confirms what I saw earlier this week: that it just doesn't update at all: image

The only log entries from homebridge-weather-plus are the hourly ones that report my sensor failures.

Now that I've gathered that data: updating to https://github.com/naofireblade/homebridge-weather-plus/commit/37409db2a05b04e2b283abad6b35533e36d64e5d

wimleers commented 4 months ago

It's now Saturday. I just opened the Eve app on my iPad for the first time in days. I see the exact same information as in the screenshot I posted in the Friday AM. The "Tomorrow" prediction is now the prediction fetched on Thursday about Friday. So it's "forecasting" yesterday 😅

dacarson commented 4 months ago

And no errors are appearing in the log?

jmissig commented 4 months ago

What's the easiest way for me to try this branch at home? I tried a git clone of @dacarson's fork and then an npm -g install from within that directory, but Homebridge complains about a bunch of missing stuff until I npm -g install homebridge-weather-plus to go back to main. I'm guessing I'm missing a step somewhere.

dacarson commented 4 months ago

I actually npm -g install homebridge-weather-plus then copy the cloned files over the top of the installed files.

dacarson commented 4 months ago

Oddly without any intervention, my tempest unit suddenly woke up again. The messages before it waking up are from my original Air and Sky units. image

wimleers commented 4 months ago

And no errors are appearing in the log?

None.

dacarson commented 4 months ago

And no errors are appearing in the log?

None.

I have been traveling, so I haven't been able to work on this till today.

Forecast stopped updating for me too. I think I know what the issue is. It is this line: if (this.lastForecastUpdate && moment().hour() != this.lastForecastUpdate) { What I wanted to happen was, if lastForecastUpdate was not defined, then don't even attempt to load the forecast. The variable lastForecastUpdate was only defined if an API key and Station ID was provided.

What I think is happening, when the hour clicked over midnight, the current hour is 0, the if statement returns false and the forecast is never updated again.

The test I am using for an undefined variable is incorrect. It looks like what I should be doing is: if ((typeof this.lastForecastUpdate !== 'undefined') && moment().hour() != this.lastForecastUpdate) { I am testing this change out.

dacarson commented 4 months ago
  • Why in the "Today" view does it say Rain probability: 20%, Condition: Rain possible (these make sense) and then Rain: Yes? I guess that's just because any non-zero chance would cause it to say Yes?

I did find an issue with this. As can be seen in my forecast below , Rain Probability: 0%, and Condition: Partly Cloudy, but it still showed Rain: Yes. I was basing the Rain Yes/No on the precipitation icon, which was incorrect as there will always be a precipitation icon even when rain/snow is not in the forecast. A better way for Rain Yes/No was to copy exactly what OpenWeatherMap is doing. They normalize the weather condition into one of the conditions that Eve understand, then they check if the condition is rain or snow. IMG_5046

dacarson commented 4 months ago

@wimleers & @jmissig Could you test this again. I fixed the issue with Forecast not updating as well as a fix around Rain in forecast.

wimleers commented 4 months ago

Updated! Will be keeping an eye on this.

P.S.: isn't it funny that the forecast seemed to be working fine for you, but then only thanks to not working on this and hence letting it run without interruption for multiple days, you were able to reproduce it? 😄

dacarson commented 4 months ago

P.S.: isn't it funny that the forecast seemed to be working fine for you, but then only thanks to not working on this and hence letting it run without interruption for multiple days, you were able to reproduce it? 😄

It was 'working for me' because there was another issue I had in my Homebridge that caused it to restart once a day. Restarting, got me past the zero hour issue, and I didn't see it initially. I fixed my restart issue, and let it run for a few days I experienced the same issue you were.

dacarson commented 4 months ago

My internet went down for a short period of time, got errors from other plugins but HomeBridge ended up restarting because of a change I made here https://github.com/naofireblade/homebridge-weather-plus/pull/285/commits/67c11a604a17f9ad9b4a9ca0e554cd384aa3e251. Patch coming

[07/03/2024, 18:04:58] [B-Hyve] Error getting API key getaddrinfo EAI_AGAIN api.orbitbhyve.com
[07/03/2024, 18:04:58] [B-Hyve] Error retrieving API key 
TypeError: Cannot read properties of undefined (reading 'status')
    at OrbitAPI.getToken (/var/lib/homebridge/node_modules/homebridge-orbit-irrigation/orbitapi.js:47:17)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at OrbitPlatform.getDevices (/var/lib/homebridge/node_modules/homebridge-orbit-irrigation/orbitplatform.js:89:24)
[07/03/2024, 18:04:58] [B-Hyve] Failed to get devices. Retry attempt 1 of 3 in 60 seconds...
[07/03/2024, 18:04:58] [B-Hyve] TypeError: Cannot read properties of undefined (reading 'user_name')
    at OrbitPlatform.getDevices (/var/lib/homebridge/node_modules/homebridge-orbit-irrigation/orbitplatform.js:90:53)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
Error: getaddrinfo EAI_AGAIN api-production.august.com
    at GetAddrInfoReqWrap.onlookupall [as oncomplete] (node:dns:118:26) {
  errno: -3001,
  code: 'EAI_AGAIN',
  syscall: 'getaddrinfo',
  hostname: 'api-production.august.com'
}
[07/03/2024, 18:05:09] [Homebridge UI] [homebridge] Failed to check registry.npmjs.org for updates: "timeout of 10000ms exceeded" - see https://homebridge.io/w/JJSz6 for help.
[07/03/2024, 18:05:09] [Homebridge UI] [homebridge-orbit-irrigation] Failed to check registry.npmjs.org for updates: "timeout of 10000ms exceeded" - see https://homebridge.io/w/JJSz6 for help.
[07/03/2024, 18:05:09] [Homebridge UI] [homebridge-weather-plus] Failed to check registry.npmjs.org for updates: "timeout of 10000ms exceeded" - see https://homebridge.io/w/JJSz6 for help.
[07/03/2024, 18:05:09] [Homebridge UI] [homebridge-orbit-irrigation] Failed to check registry.npmjs.org for updates: "timeout of 10000ms exceeded" - see https://homebridge.io/w/JJSz6 for help.
[07/03/2024, 18:05:09] [Homebridge UI] [homebridge-weather-plus] Failed to check registry.npmjs.org for updates: "timeout of 10000ms exceeded" - see https://homebridge.io/w/JJSz6 for help.
[07/03/2024, 18:05:18] [WeatherPlus] Error retrieving weather Forecast
[07/03/2024, 18:05:18] [WeatherPlus] undefined
[07/03/2024, 18:05:18] [WeatherPlus] Error getting locationGeo from London, GB
[07/03/2024, 18:05:18] [WeatherPlus] Error: getaddrinfo EAI_AGAIN api.openweathermap.org
    at GetAddrInfoReqWrap.onlookupall [as oncomplete] (node:dns:118:26) {
  errno: -3001,
  code: 'EAI_AGAIN',
  syscall: 'getaddrinfo',
  hostname: 'api.openweathermap.org'
}
[07/03/2024, 18:05:19] [Homebridge UI] [homebridge-config-ui-x] Failed to check registry.npmjs.org for updates: "timeout of 10000ms exceeded" - see https://homebridge.io/w/JJSz6 for help.
[07/03/2024, 18:05:19] [Homebridge UI] [homebridge-config-ui-x] Failed to check registry.npmjs.org for updates: "timeout of 10000ms exceeded" - see https://homebridge.io/w/JJSz6 for help.
[07/03/2024, 18:05:38] [WeatherPlus] Weather Underground Request failed
[07/03/2024, 18:05:38] TypeError: Cannot read properties of undefined (reading 'statusCode')
    at WundergroundAPI.<anonymous> (/var/lib/homebridge/node_modules/homebridge-weather-plus/apis/weatherunderground.js:73:55)
    at self.callback (/var/lib/homebridge/node_modules/homebridge-weather-plus/node_modules/request/request.js:185:22)
    at Request.emit (node:events:518:28)
    at Request.onRequestError (/var/lib/homebridge/node_modules/homebridge-weather-plus/node_modules/request/request.js:877:8)
    at ClientRequest.emit (node:events:518:28)
    at TLSSocket.socketErrorListener (node:_http_client:495:9)
    at TLSSocket.emit (node:events:518:28)
    at emitErrorNT (node:internal/streams/destroy:169:8)
    at emitErrorCloseNT (node:internal/streams/destroy:128:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)
[07/03/2024, 18:05:38] Got SIGTERM, shutting down Homebridge...
wimleers commented 4 months ago

It's been working well now! 😄

Just spotted one more (and probably/hopefully last) bug though: the name of the day for a forecast appears to be computed using UTC or the non-DST-adjusted time.

Note how for both the "day" says Saturday, even though I'm looking at the Today forecast. (Tomorrow's forecast lists Sunday at both times.)

dacarson commented 3 months ago

Just spotted one more (and probably/hopefully last) bug though: the name of the day for a forecast appears to be computed using UTC or the non-DST-adjusted time.

DST is a tricky one. Sunrise and Sunset are computed using the same algorithm. Are they off by an hour too?

forecast.ForecastDay =  moment.unix(values[i].day_start_local).tz(timezone).format('dddd');

forecast.SunriseTime = moment.unix(values[i].sunrise).tz(timezone).format('HH:mm:ss');
forecast.SunsetTime = moment.unix(values[i].sunset).tz(timezone).format('HH:mm:ss');

The time library that is being used should automatically take DST into account.

The Observation time in the screenshots are both before midnight: 22:59:36, and 23:59:36 so it thinks it is still on the same day. This will happen regardless of DST. On the next update, at 00:59:36, it should click over to the next day.

I could see if I can change the forecast fetching to the beginning of every hour rather than the end of every hour, that way as soon as it ticks over midnight, the forecast for Today, Tomorrow, etc should be correct.

dacarson commented 3 months ago

I could see if I can change the forecast fetching to the beginning of every hour rather than the end of every hour, that way as soon as it ticks over midnight, the forecast for Today, Tomorrow, etc should be correct.

Checking the code, I fetch the forecast as soon as we click over a new hour. (If the current hour has changed from the last time I fetched a forecast, then fetch a new forecast). This code is called at a 1 minute interval if you are using the Tempest weather station service. 

        if ((typeof this.lastForecastUpdate !== 'undefined')  && moment().hour() != this.lastForecastUpdate) {
            this.lastForecastUpdate = moment().hour();
... fetch forecast

I wonder if a new forecast hasn't been generated then. What I could do is if we are in a new hour, and if it is more than 2 minutes into that new hour, then fetch the forecast.

        if ((typeof this.lastForecastUpdate !== 'undefined')  && 
                (moment().hour() != this.lastForecastUpdate && moment().minute() > 2) {
            this.lastForecastUpdate = moment().hour();
... fetch forecast
wimleers commented 3 months ago

The Observation time in the screenshots are both before midnight: 22:59:36, and 23:59:36 so it thinks it is still on the same day. This will happen regardless of DST. On the next update, at 00:59:36, it should click over to the next day.

You misunderstood me: it was already wrong at 22:59:36: "Today" was still Friday! Same at 23:59:36. Only at 00:59:36 should it have said Saturday.

DST is a tricky one. Sunrise and Sunset are computed using the same algorithm. Are they off by an hour too?

No, those are correct.

I could see if I can change the forecast fetching to the beginning of every hour rather than the end of every hour, that way as soon as it ticks over midnight, the forecast for Today, Tomorrow, etc should be correct.

That shouldn't help: it was already saying "Today" was the next day between 22:00 and 23:00!

According to https://apidocs.tempestwx.com/reference/get_better-forecast the API response is of the format:

{
  "latitude": 29.00724,
  "longitude": -80.88067,
  "timezone": "America/New_York",
  "timezone_offset_minutes": -300,
  "location_name": "Home",
  "current_conditions": {
    "time": 1608668142,
    "conditions": "Partly Cloudy",
    "icon": "clear-day",
    "air_temperature": 67,
    "sea_level_pressure": 30.195,
    "station_pressure": 30.195,
    "pressure_trend": "falling",
    "relative_humidity": 55,
    "wind_avg": 3,
    "wind_direction": 15,
    "wind_direction_cardinal": "NNE",
    "wind_direction_icon": "wind-rose-sw",
    "wind_gust": 3,
    "solar_radiation": 830,
    "uv": 1,
    "brightness": 2582,
    "feels_like": 67,
    "dew_point": 67,
    "wet_bulb_temperature": 67,
    "wet_bulb_globe_temperature": 67,
    "delta_t": 10,
    "air_density": 0.08,
    "lighting_strike_count_last_1hr": 0,
    "lighting_strike_count_last_3hr": 2,
    "lighting_strike_last_distance": 7,
    "lighting_strike_last_distance_msg": "6 - 8 mi",
    "lighting_strike_last_epoch": 1608150959,
    "precip_accum_local_day": 0,
    "precip_accum_local_yesterday": 0,
    "precip_minutes_local_day": 0,
    "precip_minutes_local_yesterday": 0,
    "is_precip_local_day_rain_check": true,
    "is_precip_local_yesterday_rain_check": true,
    "includes_station_data": true
  },
  "forecast": {
    "daily": [
      {
        "day_start_local": 1608699600,
        "day_num": 5,
        "month_num": 7,
        "conditions": "Partly Cloudy",
        "icon": "clear-day",
        "sunrise": 1608639322,
        "sunset": 1608676362,
        "air_temp_high": 73,
        "air_temp_low": 54,
        "air_temp_high_color": "ff4c00",
        "air_temp_low_color": "afde06",
        "precip_probability": 10,
        "precip_icon": "chance-rain",
        "precip_type": "rain"
      }
    ],
    "hourly": [
      {
        "time": 1608735600,
        "conditions": "Partly Cloudy",
        "icon": "clear-day",
        "air_temperature": 67,
        "sea_level_pressure": 30.195,
        "relative_humidity": 55,
        "precip": 0.26,
        "precip_probability": 70,
        "precip_icon": "chance-rain",
        "wind_avg": 3,
        "wind_avg_color": "26fc59",
        "wind_direction": 15,
        "wind_direction_cardinal": "NNE",
        "wind_direction_icon": "wind-arrow-w",
        "wind_gust": 3,
        "wind_gust_color": "ebf100",
        "uv": 1,
        "feels_like": 67,
        "local_hour": 17,
        "local_day": 5
      }
    ]
  },
  "status": {
    "status_code": 0,
    "status_message": "Success"
  },
  "units": {
    "units_temp": "c",
    "units_wind": "mph",
    "units_precip": "in",
    "units_pressure": "mb",
    "units_distance": "string",
    "units_brightness": "lux",
    "units_solar_radiation": "W/m2",
    "units_other": "metric",
    "units_air_density": "kg/m3"
  },
  "source_id_conditions": 3
}

Looking at the PR, it does:

                        weather.forecasts = this.parseForecasts(result["current_conditions"]["time"], result["forecast"]["daily"], result["timezone"]);

which in turn does:

            forecast.ForecastDay =  moment.unix(values[i].day_start_local).tz(timezone).format('dddd');

Either the day_start_local value in Weatherflow's API is wrong, or the .tz() functionality in moment-timestamp is wrong, because the forecast retrieved at 22:59 today on Sunday again indicates "Monday" as the "Today" forecast 🤷

OTOH, maybe … maybe it's just that the Weatherflow forecast simply does indeed forecast the next day instead of today as we near the end of the day? So perhaps the "Today" and "Tomorrow" labels are simply wrong? 🤔 I'll add logging to see which exact timestamps are being provided by the API response, that'll clear this up! 👍

wimleers commented 3 months ago

Dumped results.forecast.daily, got this:

3/11/2024, 11:03:03 PM] [homebridge-weather-plus] [
  {
    air_temp_high: 11,
    air_temp_low: 7,
    conditions: 'Rain Likely',
    day_num: 12,
    day_start_local: 1710198000,
    icon: 'rainy',
    month_num: 3,
    precip_icon: 'chance-rain',
    precip_probability: 90,
    precip_type: 'rain',
    sunrise: 1710223507,
    sunset: 1710265509
  },
  {
    air_temp_high: 13,
    air_temp_low: 10,
    conditions: 'Rain Possible',
    day_num: 13,
    day_start_local: 1710284400,
    icon: 'possibly-rainy-day',
    month_num: 3,
    precip_icon: 'chance-rain',
    precip_probability: 40,
    precip_type: 'rain',
    sunrise: 1710309773,
    sunset: 1710352010
  },
  {
    air_temp_high: 16,
    air_temp_low: 8,
    conditions: 'Partly Cloudy',
    day_num: 14,
    day_start_local: 1710370800,
    icon: 'partly-cloudy-day',
    month_num: 3,
    precip_icon: 'chance-rain',
    precip_probability: 10,
    precip_type: 'rain',
    sunrise: 1710396039,
    sunset: 1710438511
  },
  {
…

Therefore the solution is to:

  1. verify that day_start_local corresponds to today's start of the day (or tomorrow's, or … whichever "day + N" you enabled forecasts for
  2. if no forecast is available for a certain day (e.g. today aka "day + 0"), then show the stale data if we have it or some kind of fault indicator otherwise
dacarson commented 3 months ago

I get what you are saying. I believe that the same API is used to power the website for your weather station. Mine is https://tempestwx.com/station/39778/ If you change that last number to your station ID, can you check what it shows there? Because it seems very odd that they don't have the current day forecast when you are near the end of the day. What do they show on their site at the end of the day?

wimleers commented 3 months ago

Will check tonight! 👍

dacarson commented 3 months ago

Spending more time on the website I mentioned above, they don't actually show a daily forecast for Today, they only show an hourly forecast for Today. Therefore, I am not going to learn much about what happens at 2300-0000.

So, coming back to the solution, as you said:

  1. verify that day_start_local corresponds to today's start of the day (or tomorrow's, or … whichever "day + N" you enabled forecasts for
  2. if no forecast is available for a certain day (e.g. today aka "day + 0"), then show the stale data if we have it or some kind of fault indicator otherwise

I would need to slightly change this approach. The plugin loads the maximum number of forecast days, then only chooses to show the ones you have selected. Also, the plugin doesn't maintain stale forecast data. So what I could do is just populate the days that I get a forecast for, and any day I don't get a forecast for (ie the current day that is almost over), just leave it blank.

Looking at the network requests for the web page, I have confirmed that it does use pretty much the same API to fetch the forecast, and it gets practically the same response. It fetches it from https://swd.weatherflow.com/swd/rest/better_forecast?

Patch coming

dacarson commented 3 months ago

Spending more time on the website I mentioned above, they don't actually show a daily forecast for Today, they only show an hourly forecast for Today. Therefore, I am not going to learn much about what happens at 2300-0000.

It actually does show the high/low temperature and the sunrise/sunset for the day. That said, I believe your solution still holds.

dacarson commented 3 months ago

My internet went down for a short period of time, got errors from other plugins but HomeBridge ended up restarting because of a change I made here 67c11a6. Patch coming

Patch is here (https://github.com/naofireblade/homebridge-weather-plus/pull/285/commits/d3a4f0b2cccaef044b5fe806611b476735a84bba)

dacarson commented 3 months ago

Patch to address incorrect forecasts between 11pm and midnight (https://github.com/naofireblade/homebridge-weather-plus/pull/285/commits/df066d2e3d298eadf3658c3de1e512f4169ac459) Check to make sure the forecast for today is actually today's forecast. If it is, then save a copy as I will need it later. When the forecast isn't today's, offset the forecast being stored into the forecasts array, and then insert the saved today's forecast.

Tested by running over a couple of days and I saw that the Today's forecast had a different Observation Time than In 1 Day, also the Day values where correct.

@wimleers are you up for testing the fork again :-)

wimleers commented 3 months ago

I've been logging the results.forecast.daily data. I've got enough data gathered now to confirm my earlier theory 👍

☝️ Once the last hour of the day has passed, there's no more forecast for today available. Which kinda makes sense. And I bet that there's still an hourly forecast for that last hour, but I did not check that.

dacarson commented 3 months ago

Once the last hour of the day has passed, there's no more forecast for today available. Which kinda makes sense. And I bet that there's still an hourly forecast for that last hour, but I did not check that.

You were right, thank you for debugging this. There is nothing in the API documentation about this behavior. On the website between 23:00-00:00 they drop the 'Today's' forecast and only show the forecast for the following day. This issue has now been fixed (https://github.com/naofireblade/homebridge-weather-plus/commit/df066d2e3d298eadf3658c3de1e512f4169ac459).

I wonder if OpenWeatherMap has the same issue but no one has noticed it. I'll test later today.

wimleers commented 3 months ago

Success! It's now 23:17, and your plugin is still showing Thursday for the Today forecast, with an Observation time of 22:00:02, but the Tomorrow forecast has been updated on 23:00:02 👍

Let's ship this! 🚢 🚀

dacarson commented 3 months ago

FWIW, OpenWeatherMap forecasts doesn't seem to have the same issue.

Yes, let's ship it. Thanks for all your help and support @wimleers

wimleers commented 3 months ago

Still working flawlessly here 👍😊

dacarson commented 3 months ago

Thanks @naofireblade! Just need to tag it as a new release :-)

And thanks to @wimleers for all the testing and feedback. I would like to close out as many open issues as possible so that there is a clean set of open issues remaining.

wimleers commented 3 months ago

I'll try to chip in with that effort, @dacarson 😊 I know what a challenge that can be 🤓 For context, this is the open source project I work on for my $DAYJOB, and it's the same problem, but at quite a different scale: https://www.drupal.org/about/core/blog/bug-smash-initiative-3-year-update 😄

dacarson commented 3 months ago

Thanks @wimleers All I can do is see if the issue has been fixed, or what the next steps are and add a note to the issue. In many cases the issue just needs to be closed as it behaves correctly, though I need to ask the reporter to do that.

naofireblade commented 3 months ago

I just added you as a collaborator if you want to cleanup some old issues :) .

dacarson commented 3 months ago

@naofireblade Thank you! I have started closing out old issues or issues that behave correctly.

dacarson commented 3 months ago

@naofireblade I drafted a v3.3.4-Beta release tag. I was thinking of doing a Beta this time because then people can install it and try it out. Specifically the people who have filed issues that are fixed in this release can verify that they are fixed before I move it to an official release.

naofireblade commented 3 months ago

Published to npm as 3.3.4-beta.1 Install with npm install -g homebridge-weather-plus@beta