Grab min max from OWM daily like precip?

[jeroenterheerdt]

What happened?

See title

How to reproduce

See title

Relevant log output

No response

Which version are you running?

Beta

Diagnostics file

• [X] I have attached a diagnostics file

Additional information

No response

[jeroenterheerdt]

Thanks, the timestamps should help. Note that the timestamps are not just on weatherdata but also on mappings and zones.

[dlindnegm]

I don't see any "retrieved" timestamps for each update for my sensors, just the OWM updates. It does show the data_last_updated at the end, just not for each hourly update.

OWM: { "Windspeed": 0.628278903141073, "Pressure": 1016, "Humidity": 72, "Temperature": 18.53, "Dewpoint": 13.39, "Precipitation": 4.1, "retrieved": "2023-09-09T08:50:01.177685" } ], "data_last_updated": "2023-09-09T08:50:01.180663" } My Sensors: { "Dewpoint": 19.222222222222218, "Humidity": 93.0, "Precipitation": 0.0, "Pressure": 977.8201125, "Solar Radiation": 5.040576000000001, "Temperature": 20.500000000000004, "Windspeed": 0.0 } ], "data_last_updated": "2023-09-09T08:50:01.183119" }

[andreid1303]

I'm apologise for my flood without knowing how exactly FAO-56 Penman-Monteith method works. After close look into pyETO I found that in case we "Estimate Solar radiation" only daily calculations are correct.

Also about OWM OneCall data: what "daily" data are you discussing? Per [https://openweathermap.org/api/one-call-3#current] there is currently observed data and forecast: minutely, hourly and daily.

No max, min, mean in current:

"current": {
        "dt": 1694266438,
        "sunrise": 1694227821,
        "sunset": 1694274741,
        "temp": 15.3,
        "feels_like": 13.98,
        "pressure": 1017,
        "humidity": 42,
        "dew_point": 2.48,
        "uvi": 0.29,
        "clouds": 100,
        "visibility": 10000,
        "wind_speed": 5.9,
        "wind_deg": 16,
        "wind_gust": 9.58,
        "weather": [
            {
                "id": 804,
                "main": "Clouds",
                "description": "overcast clouds",
                "icon": "04d"
            }
        ]
    },

OWM data updated every 10 minutes and suggests collecting data as frequently as possible (for correct aggregates in our case). Free 1000 calls in a day is 1 minute limit if you don't use other OWA integrations. I think 5-10 minutes is optimum.

[jeroenterheerdt]

I'm apologise for my flood without knowing how exactly FAO-56 Penman-Monteith method works. After close look into pyETO I found that in case we "Estimate Solar radiation" only daily calculations are correct.

Also about OWM OneCall data: what "daily" data are you discussing? Per [https://openweathermap.org/api/one-call-3#current] there is currently observed data and forecast: minutely, hourly and daily.

No max, min, mean in current:

"current": {
        "dt": 1694266438,
        "sunrise": 1694227821,
        "sunset": 1694274741,
        "temp": 15.3,
        "feels_like": 13.98,
        "pressure": 1017,
        "humidity": 42,
        "dew_point": 2.48,
        "uvi": 0.29,
        "clouds": 100,
        "visibility": 10000,
        "wind_speed": 5.9,
        "wind_deg": 16,
        "wind_gust": 9.58,
        "weather": [
            {
                "id": 804,
                "main": "Clouds",
                "description": "overcast clouds",
                "icon": "04d"
            }
        ]
    },

OWM data updated every 10 minutes and suggests collecting data as frequently as possible (for correct aggregates in our case). Free 1000 calls in a day is 1 minute limit if you don't use other OWA integrations. I think 5-10 minutes is optimum.

When I say daily I mean daily.

[jeroenterheerdt]

I don't see any "retrieved" timestamps for each update for my sensors, just the OWM updates. It does show the data_last_updated at the end, just not for each hourly update.

OWM: { "Windspeed": 0.628278903141073, "Pressure": 1016, "Humidity": 72, "Temperature": 18.53, "Dewpoint": 13.39, "Precipitation": 4.1, "retrieved": "2023-09-09T08:50:01.177685" } ], "data_last_updated": "2023-09-09T08:50:01.180663" } My Sensors: { "Dewpoint": 19.222222222222218, "Humidity": 93.0, "Precipitation": 0.0, "Pressure": 977.8201125, "Solar Radiation": 5.040576000000001, "Temperature": 20.500000000000004, "Windspeed": 0.0 } ], "data_last_updated": "2023-09-09T08:50:01.183119" }

That's a mistake on my side, apparently added the retrieved only on OWM data.

[andreid1303]

Like that?:

"daily": [
    {
        "dt": 1694250000,
        "sunrise": 1694227821,
        "sunset": 1694274741,
        "moonrise": 0,
        "moonset": 1694267640,
        "moon_phase": 0.83,
        "summary": "Expect a day of partly cloudy with rain",
        "temp": {
            "day": 14.25,
            "min": 10.1,
            "max": 15.7,
            "night": 10.1,
            "eve": 15.18,
            "morn": 11.07
        },
        "feels_like": {
            "day": 12.93,
            "night": 8.6,
            "eve": 13.88,
            "morn": 10.3
        },
        "pressure": 1016,
        "humidity": 46,
        "dew_point": 2.63,
        "wind_speed": 7.74,
        "wind_deg": 12,
        "wind_gust": 9.78,
        "weather": [
            {
                "id": 500,
                "main": "Rain",
                "description": "light rain",
                "icon": "10d"
            }
        ],
        "clouds": 56,
        "pop": 0.44,
        "rain": 0.66,
        "uvi": 2.14
    },

[dlindnegm]

In addition to no "retrieved" timestamps being written to my local sensor updates, I just noticed that the "Automatic weather data update" doesn't survive a restart of HA. All of my hourly updates stopped after I restarted HA this morning for the 2023.9.1 Core update. So I won't have any hourly data until tonight's calculation is performed. Hopefully there's a way to restart that after a HA restart?

[jeroenterheerdt]

Like that?:

"daily": [

yes.

[jeroenterheerdt]

In addition to no "retrieved" timestamps being written to my local sensor updates, I just noticed that the "Automatic weather data update" doesn't survive a restart of HA. All of my hourly updates stopped after I restarted HA this morning for the 2023.9.1 Core update. So I won't have any hourly data until tonight's calculation is performed. Hopefully there's a way to restart that after a HA restart?

you will have hourly data, it has been collected all day for you by the time the calc is performed. I will look into what's happening here. It should be persisted between restarts. I'll open a separate bug because this thread becomes too confusing with all things in here.

[andreid1303]

Like that?:

"daily": [

yes.

But this is "daily forecast" with first timestamp of this day at 09:00:00 GMT+0000.

"current": { "dt": 1694280488, Sat Sep 09 2023 17:28:08 GMT+0000 "daily": [ { "dt": 1694250000, Sat Sep 09 2023 09:00:00 GMT+0000

Although it is updated through the day, it is still forecast. For example, for me two hours ago tmin was 10.1C, now it is 10.7C. But daily minimum temperature can't rise by definition.

[jeroenterheerdt]

You're going to come to the same conclusion I have a while back and why I started v2. Look back at this thread and you will see that we have stopped getting data from daily except for precipitation as there is no reliable way to tell that from current. Unless you have a good alternative for that that works based on OWM I think the discussion you're trying to have here is irrelevant as we have already moved passed it. Did you read the other messages in this thread?

[andreid1303]

Yes, I read, no found any reference that you were tring to use forecast data. No mind.

[andreid1303]

Going to use this https://github.com/petergridge/openweathermaphistory. I don't see problems with getting current precipitation with frequent updates.