Precipitation Today

[jazzyisj]

Precipitation accumulation is available as forecast sensors for hourly/daily, but there is no real time entity provided for precipitation today. Because the forecast sensors do not have a state_class, they cannot be used for statistics.

Possible to add an entity for precipitation today with the appropriate state_class so it can be utilized by statistics?

Also, there IS an entity for precipitation intensity, but it has not been assigned a state_class.

[alexander0042]

This should be pretty quick! Clarification question though- when you're asking about "precipitation today", do you mean the total precipitation that fell today? Or the remaining precipitation for today? Or something else?

[jazzyisj]

I am referring to the precipitation that has actually accumulated for the current day. I think precipitation in last 24 hours would probably be even more useful, but I believe precipitation accumulation is only given for the current day by most weather providers? I know my Weatherflow station resets the precipitation at midnight and Environment Canada only provides the total precipitation for the previous day. Looks like accuweather does provide last 24 hours though (as well as last hour, last 3 hours etc). I'm not sure where the data for pirateweather is coming from.

https://developer.accuweather.com/accuweather-current-conditions-api/apis/get/currentconditions/v1/%7BlocationKey%7D

Either way I think it would be a welcome addition to PirateWeather. Thanks for looking into it!

[jhemak]

+1 for this

[alexander0042]

Been thinking a bit about this, and what each provider does is kind of all over the place. Apple Weather seems to only show the remaining precipitation, while you're rights, others show the total for the day. Unfortunately, this is where some of the "art" of putting together a weather API comes in, since there's no correct answer.

I try to think of someone opening a weather app on a phone and seeing 12 mm of precipitation on the current day when they check the weather at 8 am, after a rainy night but then being surprised because it doesn't rain during the day; so that's how I came to the current setup.

Luckily, since this started out as a Home Assistant question, there's an easy enough fix on that front. If you create a sensor in HA following this comment, it'll always show the measured + foretasted accumulation for the current day as the "day 1" (not day 0) data point, with the sum of the first 24 hourly forecasts being the trailing 24 hour precipitation. Every since the v2.0 update this has been based on observed conditions, so should be an accurate way to go about this!

[jhemak]

Hey @alexander0042, thanks a ton for this info! I've read and re-read and must be missing something. How come you are having us look at day 1 not day 0 for the measured + forecasted accumulation for the current day? I thought current day = day 0?

@cloneofghosts, would there be a way to get this sensor in HA without an additional/separate call to the API?

[alexander0042]

Great question! If you make a request for time=-86400, then it's asking the API for the weather 24 hours ago. So if you make the request on Tuesday at 7:00 using time=-86400, then the API returns the forecast starting from Monday at 7:00. This means that day 0 is Monday, so day 1 would be precipitation for the current day (Tuesday).

Adding a sensor that way is an extra call, which isn't ideal. For what it's worth, if you're only looking for one location and making a request every 15 minutes, then you'd still be well within the free tier limit, even with the two calls! The other option would be to use a template instead of the integration and make every request -86400, which would shorten the forecast by one day, but still work

[cloneofghosts]

If I were to query the API using time=-86400 right now for the current day would I see the forecasted precipitation from yesterdays NBM 2PM model run or would it be a combination of today's HRRR 0h data and yesterdays NBM 2PM model run?

[alexander0042]

This was the big change in V2.0, and it was a pain and a ! You'd see merged NBM 1-hour forecasts until the current forecast time (Todays's 2 pm run), and then that from there on out. So to clarify:

Request time: 5 pm Tuesday, requesting -86400

• Hour 0 (5 pm Monday): 1-hour forecast from the 4 pm NBM run
• Hour 1 (6 pm Monday): 1-hour forecast from the 5 pm NBM run
• Hour 2 (7 pm Monday): 1-hour forecast from the 6 pm NBM run ...
• Hour 20 (2 pm Monday): 1-hour forecast from the 1 pm NBM run
• Hour 21 (3 pm Monday): 1-hour forecast from the 2 pm NBM run
• Hour 22 (4 pm Monday): 2-hour forecast from the 2 pm NBM run
• Hour 23 (5 pm Monday): 3-hour forecast from the 2 pm NBM run

And so on and so forth. This provides a really nice blend of latest results with the most accurate data, while also keeping the file sizes small on the back end. Essentially, the ingest script queries 36 hours worth of 1-hour forecasts from the archive and merges then together in the operational file, producing a single time series

[cloneofghosts]

Ah ok that makes sense. So the solution here would be to use a template sensor to get the data for the current day?

[cloneofghosts]

@alexander0042 Just looking through this again and I think there's been some confusion as to what @jazzyisj is looking for. They were looking for a data point which shows the total precipitation accumulation so far for the current day.

So if you queried the API at 7am the field would show the total amount of precipitation which accumulated between 12am and 7am.

The other option was to have a field which showed the total amount of precipitation which accumulated in the last 24h. Again if I were to query the API at 7am it would show the total accumulation between 7am yesterday and 7am today.

[jazzyisj]

@cloneofghosts - Yes, precisely what I was hoping for! Right now I'm grabbing precipitation accumulation data from my weather station, but it doesn't measure snow which I'm hoping Pirate Weather will be able to someday. (I'm super excited to see how the rain/snow/ice forecasts pan out btw!)

I am using a non_negative_sum statistics helper to convert accumulation today to accumulation in the last 24 hours (the weather station resets at midnight) and another one to break out accumulation in just the last hour.

If you're curious, this is how I use all these sensors in my weather ui.

The single Precip Accum graph breaks the day into 24 segments and uses the accum last hour sensor to visualize when and how much rain each hour in the last 24 hours. (DItto with the Precip Time graph beside it, how long it rained each hour.)

Precipitation detail and weekly graph. Daily precipitation is tracked in the graph.

And some statistics stuff.

[alexander0042]

What a cool use case! I can't get over how nice that dashboard looks, and would love to compare to a personal weather station.

The tricky thing is where to put this variable. I can see how it would be useful, and already calculate it, so want to put it in, but also don't want to break existing setups. My idea (going on the dev backend soon) is to update the version tag to allow 2.4, and to put it in the currently section then. That way it shouldn't throw off anyone who has the currently section hard-coded, but also accessible.

[cloneofghosts]

Couldn't the variable be hidden behind the version=2 flag which already exists since that's where the new variables added in V2 are located?

[alexander0042]

It could, but since V2 has been out for a bit, I'm a little worried about it causing issues. The flags section is usually where things get added, since that's a bit of a grab bag; however, the currently section is way more widely used, so higher chance of an issue.

Maybe I'm being paranoid though, since I really think most clients should be able to handle new fields without issue!

[cloneofghosts]

I would think it would be fine to add it under the V2 flag as I think most clients should be able to grab the data without issue. It's your API and if you'd rather put it under a new flag then that's fine.

If a version=2.4 (or whatever the version number is) flag was added it would still show the data from the V2 flag correct?

[alexander0042]

I'm sold! It's live on the dev backend in using the regular "version=2" flag, with the fields: -currentDayRain -currentDaySnow -currentDayIce

[cloneofghosts]

@alexander0042 Maybe this is something for the docs but are those fields in mm or cm? I would think it would be in cm to match the other accumulation fields but it seems like its in mm like the intensity fields.

I also noticed that on the daily block most of the daily time fields return -999 instead of an epoch timestamp. I've included a daily block for my location which shows the issue:

{
  "time": 1729742400,
  "icon": "clear-day",
  "summary": "Clear",
  "dawnTime": 1729767664,
  "sunriseTime": 1729769515,
  "sunsetTime": 1729807279,
  "duskTime": 1729809128,
  "moonPhase": 0.76,
  "precipIntensity": 0,
  "precipIntensityMax": 0,
  "precipIntensityMaxTime": -999,
  "precipProbability": 0,
  "precipAccumulation": 0,
  "precipType": "none",
  "temperatureHigh": 10.23,
  "temperatureHighTime": -999,
  "temperatureLow": 0.35,
  "temperatureLowTime": -999,
  "apparentTemperatureHigh": 5.72,
  "apparentTemperatureHighTime": -999,
  "apparentTemperatureLow": -2.37,
  "apparentTemperatureLowTime": -999,
  "dewPoint": 1.3,
  "humidity": 0.68,
  "pressure": 1018.83,
  "windSpeed": 12,
  "windGust": 19.67,
  "windGustTime": -999,
  "windBearing": 312.5,
  "cloudCover": 0.3,
  "uvIndex": 3.42,
  "uvIndexTime": -999,
  "visibility": 16.09,
  "temperatureMin": 3.16,
  "temperatureMinTime": -999,
  "temperatureMax": 10.23,
  "temperatureMaxTime": -999,
  "apparentTemperatureMin": 0.6,
  "apparentTemperatureMinTime": -999,
  "apparentTemperatureMax": 5.72,
  "apparentTemperatureMaxTime": -999,
  "smokeMax": 0,
  "smokeMaxTime": -999,
  "liquidAccumulation": 0,
  "snowAccumulation": 0,
  "iceAccumulation": 0,
  "fireIndexMax": 17.97,
  "fireIndexMaxTime": -999
},

[alexander0042]

And this is why I made the dev endpoint! To answer the two questions:

1. Units should be the same as the regular accumulation, so mm/ cm. As a check, the accumulation for a request with the time set to midnight of day 0 should equal the "accumulation currentDay" plus the daily day 0 accumulation for any other time.
2. This is a bug in the fix to the global invalid data filter from the UV issue. Since the UNIX times are larger than the 10,000 cutoff, it's filtering them out. I'll correct that and push an update!
3. There seems to be some sort of other error where precipitation isn't showing up for the dev endpoint, so also investigating that

[cloneofghosts]

I thought that the accumulation units were cm everywhere as that is what Dark Sky used. Is that no longer the case?

[alexander0042]

Apparently I need more coffee- should have said cm/ inches! Sorry for the confusion there!

Also, fixed the time and accumulation bugs. Just need to find somewhere where it's already rained to double check the day 0 accumulation

[cloneofghosts]

More coffee is always good :smile:. When I checked earlier Yellowknife was showing Snow at the airport so should be a good location to check.

Just checked and the daily time variables issue has been fixed. The only other issues I see are: the alert region locations still have a leading space and the max-age is currently set to 3600 seconds (1 hour) when it should be 900 (15 minutes).

[alexander0042]

Totailly forgot about the leading whitespace! Thanks for reminding me.

Just put those two changes in, and docs update coming this afternoon

[cloneofghosts]

@alexander0042 Just noticed a couple more issues on dev that need to be fixed:

1. Elevation is always 0
2. I noticed some instances of precipIntensity not being rounded to 4 decimal points. A location to test with is Whitehorse (60.727,-135.074). Seems to be mainly isolated in the hourly blocks.

[alexander0042]

The intensity rounding was a tricky one, but got that! Elevation was just a dev instance thing, and also corrected

[cloneofghosts]

Just checked and everything looks good.

[cloneofghosts]

@alexander0042 The dev endpoint if you don't have the version=2 query string in the URL you get an Internal Server Error.

[alexander0042]

That would have been an issue! Got it fixed

[jazzyisj]

I've finally had a chance to look at the data for the new current day liquid (rain) accumulation.
Posting this here in case anyone has the same issue I did.

This issue is probably out of pirate weather control, but the value for sensor.pirateweather_current_day_liquid_accumulation decreases sometimes. This means it can't be used directly with a statistics helper using the sum_differences_nonnegative as I described here to discern precipitation in the last 24 hours, last hour etc. from the current day accumulation value.

You can still use the data, it just has to be filtered using a trigger template sensor to prevent decreasing values messing with your statistics helpers. You'll have to create an number helper named Liquid Accumulation Today Checked and the following template sensor and then use sensor.rain_accumulation_today as the entity for your statistics helpers.

- trigger:
    - trigger: homeassistant
      event: start

    - trigger: event
      event_type: event_template_reloaded

    - trigger: state
      entity_id: sensor.pirateweather_current_day_liquid_accumulation
      to:
  action:
    - variables:
        previous: "{{ states('input_number.liquid_accumulation_today_checked') | float(-1) }}"
        current: "{{ states('sensor.pirateweather_current_day_liquid_accumulation') | float(-1) }}"

    - action: input_number.set_value
      target:
        entity_id: input_number.liquid_accumulation_today_checked
      data:
        value: "{{ 0 if current == 0 else iif(current > previous, current, previous) }}"
  sensor:
    - name: Rain Accumulation Today
      unique_id: rain_accumulation_today
      icon: mdi:weather-pouring
      device_class: precipitation
      state_class: measurement
      unit_of_measurement: mm
      state: "{{ states('input_number.liquid_accumulation_today_checked') }}"