search

Pirate-Weather / pirateweather

Code and documentation for the Pirate Weather API
Apache License 2.0
671 stars 30 forks source link

API Version 2, Beta testing notes, issues, and feedback #170

Closed alexander0042 closed 7 months ago

alexander0042 commented 8 months ago

Describe the issue

Excited to announce that the long awaited version 2.0 of the Pirate Weather API is ready for beta testing on the dev.pirateweather.net endpoint! Ever since issue #5 was opened two years ago, I realized that eventually my back-end would need a rewrite to handle a wider range of cases, and this is the culmination of that effort. I'll have a more detailed write-up eventually, but for now, I wanted to start this thread to consolidate issues with the new API in one place for me to change, as well as announce some of the new features.

At a high level, there are four main improvements that will impact every request:

  1. Includes a fancy new model from NOAA called the National Blend of Models
  2. Does a way better job of calculating the daily high/ low/ accumulations for the current day
  3. Returns somewhere between 10 and 50 times faster than v1 (this was my favourite to work out). I can't say for sure that it's the fastest weather API out there, but it's definitely in contention now.
  4. Faster data ingest (~5 minutes).
  5. Improved US alert processing.
  6. Nearest storm distance and bearing!

As well as several new optional improvements behind a new version=2 querystring parameter, to avoid breaking Dark Sky compatibility:

  1. Short term (~36 hour) air quality forecasts (top requested feature!).
  2. Liquid, snow, and ice precipitation types.
  3. Model specific exclusions (exclude=hrrr or exclude=nbm), to facilitate performance comparisons between models.
  4. Returned grid indexes of model results (this seemed small, but since HRRR is in Lambert, it was fairly complex).

I can't say enough how much the community support for this project has made this possible. Between the feedback, corrections, suggestions, and contributions, this project has gone from from a way for me to keep my smart mirror working to a community driven effort to make accurate, compatible, and documented weather model data easily available to developers around world! In particular, I can't say thank you enough to @cloneofghosts, who's taken the lead on keeping the repositories humming along and the Home Assistant integration improving while I've focused on this.

If anyone has a minute, give it a spin (every existing key should work), and let me know how it's performing.

Acknowledgements

alexander0042 commented 8 months ago

As an first note, here are my "known bugs" to get things started, in addition to the list over at #155:

cloneofghosts commented 8 months ago
  • Daily high/low seems to be a day behind where it should be

I think it's actually a day ahead of where it should be. The hourly section for me shows a maximum temperature of 9.79 and the daily high shows 11.08.

Taking a look at the V2 features I don't really see anything which should break DarkSky compatibility as they are all new fields but I guess you would rather play it safe?

I would also hazard a guess that you haven't looked at seeing if you can use the existing translation module to add in translations without doing the summaries?

EDIT: Seems like hrrr_subh and hrrr_0-18 don't have the same sourceTimes and are off by 1 or 2 hours.

Monstergerm commented 8 months ago

I think the daily0 temperature high (Today's High) is actually yesterday's High (as of 10 am local time). The hourly forecast and daily0 temperate max seem to be right.

alexander0042 commented 8 months ago

Thanks for consolidating all that! Just pushed Rev C out, which should fix all the daily max/min timing issues (turns out there were several different things that weren't working), along with most of the #155 items!

Couple other points:

Here's a question: should precipitation accumulation for day 0 (the current day) be the total for the day, or the total remaining for that day. So if it's 2 pm and it's already snowed 5 cm, with another 2 cm in the forecast, would it make more sense to return 2 cm (how it currently works), or 7 cm (how the temperature max/min will work)?

cloneofghosts commented 8 months ago

Thanks for consolidating all that! Just pushed Rev C out, which should fix all the daily max/min timing issues (turns out there were several different things that weren't working), along with most of the #155 items!

Feel free to mark any items in that issue as complete as you fix them. I think I marked any ones that I noticed as complete after I saw your comment.

  • HRRR subhourly and HRRR 0-18 will always be 1-2 hours off now. I'm trying to ingest the subhourly data as fast as possible, so it gets read as soon as four timesteps are published. The hourly data waits until the 18 hour timestep is out, and then has to process the previous 36 hours of data to create a cohesive timeseries.

Makes sense and thanks for clarifying.

  • The translation module will be easy to add in, but the harder part is writing it in English first. It's just a giant if statement, so very doable, but that's for later.

I mentioned it would be nice for V2 so totally understandable it's not high on the priority list.

  • Some libraries don't like it when I add additional fields to the json output, hence hiding the new variables behind a v2 flag. I don't know how big of an issue this is though, so maybe doesn't matter.

I haven't directly used any libraries myself but I would think it would skip over any fields instead of throwing an error but I understand the caution . What you could always do is initially keep them behind a flag but after a certain amount of time remove the flag and add it to the base API. Would give any libraries time to implement the changes without breaking current compatibility.

Here's a question: should precipitation accumulation for day 0 (the current day) be the total for the day, or the total remaining for that day. So if it's 2 pm and it's already snowed 5 cm, with another 2 cm in the forecast, would it make more sense to return 2 cm (how it currently works), or 7 cm (how the temperature max/min will work)?

To me it would make more sense to keep it how it currently works and as far as I know it's how most APIs handle things.

EDIT: I see the hourly and possibly daily summaries still sometimes show a different icon/summary

"hourly": {
"summary": "Partly Cloudy",
"icon": "cloudy",

I noticed his with cloud cover but it seems like some of the currently data initially pulls in data from the next hour at the start of the hour then interpolates to the current hour values when it should be the other way around.

cloneofghosts commented 8 months ago

@alexander0042 Something broke with the icon/summary calculation on the currently section. Today no matter what the currently cloud cover showed the icon/summary was always clear. When there is any precipitation or when precipitationProbability is over 30% (even if there is no precipitation currently) it shows None.

Everything used to work without issue so probably something in the 2.0c update broke the icon/summary calculation on the currently block.

alexander0042 commented 8 months ago

Great catch- it was fixing the cloud cover percentages in the currently section, I forgot to update the icon thresholds from 75 to 0.75! Fixed it now, and should see the changes in the next 15 minutes or so

cloneofghosts commented 8 months ago

Did the precipitation issue also get fixed? If there is any precipitation or if the precipitationProbability is over 30% the currently section displays None. A location to test this with is Winnipeg or (49.900501,-97.139313)

Also this as referenced in my comment above:

I noticed his with cloud cover but it seems like some of the currently data initially pulls in data from the next hour at the start of the hour then interpolates to the current hour values when it should be the other way around.

EDIT: Seems like the icon/summary isn't fully fixed as it keeps switching back to Clear and then back to the proper condition. This is what I see as the summary in HA (I only update the integration every 30m):

image

EDIT2: Sorry for asking a million questions but I have a few more questions in addition to the above:

  1. You mentioned that you process the HRRR subhourly data after 4 data steps so I thought that it would mean that it would mean that the subhourly data would appear in the API faster but it appears that isn't the case.
  2. Is the precipitation data still coming from the HRRR hourly or does the subhourly have that data as well? Just curious because it missed a ~2h storm at approx. 7pm but did get the later storm without issue. (To be fair the live API didn't show anything either so it's not just a development endpoint issue)
alexander0042 commented 8 months ago

Don't apologize for the question, they're what help the API to improve, so keep em coming! The precipitation type issue is a bug with my GEFS interpolation, fixing it today!

With respect to SubHourly, it runs out four hours, so 16 steps, and should be ingested faster than the hourly HRRR, so let me know if the times aren't showing that. In terms of sourcing, assuming we're in the right coverage areas (writing this up so I have a note for the docs): Currently:

Minutely

Hourly/ Daily

The logic behind this is that subhourly should be more accurate below an hour; however, I don't have a probability from it, so fall back to NBM for that. Part of the issue is when NBM shows rain, but HRRR doesn't, since there isn't an intensity or type. Conversely, when HRRR has precipitation, but probability in the NBM,then there's rain with 0 probability. What V1 does is set it to zero if either of those show zero, which is a clean and easy solution, but runs the risk of missing something.

cloneofghosts commented 8 months ago

With respect to SubHourly, it runs out four hours, so 16 steps, and should be ingested faster than the hourly HRRR, so let me know if the times aren't showing that. In terms of sourcing, assuming we're in the right coverage areas (writing this up so I have a note for the docs): Currently:

  • Intensity: subhourly
  • Type: subhourly
  • Probability: NBM

Minutely

  • Intensity: subhourly
  • Type: subhourly
  • Probability: NBM
  • Error: GEFS

Hourly/ Daily

  • Intensity: NBM
  • Type: NBM
  • Probability: NBM
  • Error: GEFS

Thanks for that information it's very helpful. I haven't noticed the sub-hourly integrating any faster than it was before but I don't pay too much attention to the sourceTimes.

The logic behind this is that subhourly should be more accurate below an hour; however, I don't have a probability from it, so fall back to NBM for that. Part of the issue is when NBM shows rain, but HRRR doesn't, since there isn't an intensity or type. Conversely, when HRRR has precipitation, but probability in the NBM,then there's rain with 0 probability. What V1 does is set it to zero if either of those show zero, which is a clean and easy solution, but runs the risk of missing something.

I'm little bit confused as to what you said there but my takeaway is that sometimes NBM forecasts precipitation with no intensity or type and in that case you set the type to none and the intensity to 0? For the currently/minutely sections if HRRR is showing precipitation but NBM has a 0% chance of precipitation then it also gets set to 0?

In the second case my preference would be that the minutely/currently data show precipitation even if the percent chance is 0. Not sure how other APIs handle things but that would make the most sense to me.

So you are fixing the currently icon/summary points today or is that an unrelated issue? Is the interpolation issue where it starts from the next hour instead of the current one also going to be fixed today or will that be fixed afterwards?

alexander0042 commented 8 months ago

Yea, great comments here! I put the changes in at lunch, and so most of these issues should be resolved in version "e". The missing icons was a separate problem, but should also be fixed.

The method I described in the previous comment started getting super complex, so I simplified it along the lines of what you suggested. It means that probability and intensity are not always aligned, but I think that it's actually not that big of a problem, and brings the results closer to the raw model (which is always good). Now what happens:

Currently/ Minutely

Hourly/ Daily

So type always comes from the same source as intensity, which should avoid most of that weirdness. Moreover, unless people specifically exclude NBM, then most people will either get NBM or GEFS, both of which give me probability and intensity, so fewer inconsistencies as well. The only remaining weirdness is currently/, minutely, but probability also makes the least sense for those ones, and isn't used as often in applications. i.e. it makes more sense to say there's a 50% chance that it will rain tomorrow than a 50% chance it's currently raining

cloneofghosts commented 8 months ago

I'm not entirely sure why Dark Sky decided to have precipitationProbability on the currently section as it doesn't really make a whole lot of sense (to me having it on minutely doesn't make sense either but maybe it's useful to someone). I'm also kinda surprised that HRRR hasn't been showing any snow as it's been snowing for a while with some big flakes too.

The currently summary/icon doesn't show rain/snow/sleet if there's any precipitation so that'll need to be fixed.

Has the interpolation issue with the currently data been fixed at all? From what I can tell the datapoints which use NBM as their source start the top of the hour using the data from the next hour and then interpolate to the current hours data. An example would be:

hour 0 cloud cover is 0.77 hour 1 cloud cover is 0.88 start of the hour cloud cover is 0.88 end of the hour cloud cover is 0.77

image

It also might be a good thing to document which variables are coming from which model for the currently conditions. I have my ideas on what is coming from where but I could be wrong.

alexander0042 commented 8 months ago

Looking into the cloud cover, but re: the icon, can you elaborate on that? It's looking correct on my end (ex. 48,-84.5), but might be something else

cloneofghosts commented 8 months ago

I think visibility is also affected by the same bug as cloud cover. The other bug may have already been fixed but last night I noticed that Montreal was saying Cloudy when the precipIntensity was 6. I had to search around to find a place where it's raining/snowing. I tested Owen Sound but it seems to work fine.

EDIT: OH I think I figured out what the issue was. I think the precipProbability on the currently section was below the threshold to show the precipitation icon which is why it showed Cloudy instead of Rain.

alexander0042 commented 8 months ago

Got it! You're right that it's a linear interpolation between hours 0 and 1 but I had the interpolation factors backwards 😒, hence the sudden jumps every hour. Just pushed out f, which addresses this

cloneofghosts commented 8 months ago

Just tested and the fix didn't fix the issue. Still seeing it jump from the current hour to the next hour at the start of the hour and it's still interpolating backwards.

EDIT: I also added the last few bugs from issue #155 in the comment where I added links to the open issues. Probably helpful if everything is in one place.

I also see that the interpolation issue has been fixed.

cloneofghosts commented 8 months ago

@alexander0042 In addition to the above comment I noticed on the development API that the models are integrating slower than they are on the live API.

Development:

"sourceTimes": {
"hrrr_subh": "2024-03-25 16Z",
"hrrr_0-18": "2024-03-25 15Z",
"nbm": "2024-03-25 15Z",
"hrrr_18-48": "2024-03-25 12Z",
"gfs": "2024-03-25 12Z",
"gefs": "2024-03-25 06Z"
},

Live:

"sourceTimes": {
"hrrr_0-18": "2024-03-25 17:00:00",
"hrrr_subh": "2024-03-25 17:00:00",
"hrrr_18-48": "2024-03-25 12:00:00",
"gfs": "2024-03-25 12:00:00",
"gefs": "2024-03-25 06:00:00"
},

I know that with the change HRRR 0-18 should be slower than on live but the subhourly HRRR is also 1h behind on the development API when compared to live.

cloneofghosts commented 8 months ago

@alexander0042 One thing I've noticed is that the API can show a precipitation icon/summary in the currently icon with precipIntensity being 0.

Not sure if it's related to the issue with precipitation I pointed out earlier (and have an example in #180) or if it's just that the instensity is so small that because of rounding it doesn't show any value though the minutely data isn't showing any precipitation for 28 minutes.

"currently": {
  "time": 1712154480,
  "summary": "Rain",
  "icon": "rain",
  "nearestStormDistance": 0,
  "nearestStormBearing": 0,
  "precipIntensity": 0,
  "precipProbability": 0.25,
  "precipIntensityError": 0.47,
  "precipType": "rain",
  "temperature": 5.42,
  "apparentTemperature": -0.15,
  "dewPoint": -1.55,
  "humidity": 0.65,
  "pressure": 996.45,
  "windSpeed": 24.68,
  "windGust": 56.19,
  "windBearing": 96,
  "cloudCover": 0.9,
  "uvIndex": 0.24,
  "visibility": 13.85,
  "ozone": 396.59
},

Now I'm seeing the None icon/summary issue which I thought was already fixed:

"currently": {
  "time": 1712163660,
  "summary": "None",
  "icon": "none",
  "nearestStormDistance": 0,
  "nearestStormBearing": 0,
  "precipIntensity": 0,
  "precipProbability": 0.5,
  "precipIntensityError": 0.67,
  "precipType": "none",
  "temperature": 7.3,
  "apparentTemperature": -0.29,
  "dewPoint": -2.3,
  "humidity": 0.71,
  "pressure": 995.77,
  "windSpeed": 30.78,
  "windGust": 58.04,
  "windBearing": 97,
  "cloudCover": 0.91,
  "uvIndex": 0.66,
  "visibility": 7.86,
  "ozone": 389.17
},
alexander0042 commented 8 months ago

Back at this! To answer your question about ingest times, it likely has something to do with the nuances of the processing triggers, but I'm not entirely sure. If you notice it again can you also write down the current time, since then I can go back and see what's happening here.

With respect to currently precipitation, it's because the logic is:

if probability is greater than 25%:
    show the precipitation type

The issue is that since (for currently) the probably comes from NBM and the type from HRRR, these can sometimes disagree. To fix this, I'm thinking of switching it to a threshold approach, where it's something like:

if precipitation rate is greater than 0.25 mm/h:
    show the precipitation type

That way they'll both be from HRRR (or GFS outside of North America), which should fix it (and probably be more accurate).

cloneofghosts commented 8 months ago

Ingest Times - Outside of that one time I haven't really noticed a difference between the live API and the dev API.

Using a threshold instead of using probabliilty makes sense as long as everything is consistent between the different data points. I didn't know this until recently but the DarkSky translation module actually lists the threshold for the different intensites.

DarkSky listed "very light" precipitation as 0-0.25mm/h so it sounds like they're showing an icon for any intensities but I don't mind if 0.1 is used instead of 0.

alexander0042 commented 8 months ago

Just to go over the tasks from #180, I want to make sure I hit them all:

Tasks

I need to test out 2.0h a bit more before releasing, but you should see it tomorrow!

cloneofghosts commented 8 months ago

Just to go over the tasks from #180, I want to make sure I hit them all:

Tasks

  • [ ] visibility is not capped ** Can you recall what it should be capped at?

16.09 km (or 10 miles)

  • [ ] minutely block seems to show array positioning ** What do you mean by this?

This is what I see in the data section in the minutely block:

"0": {
  "time": 1712176260,
  "precipIntensity": 2.1583,
  "precipProbability": 0.79,
  "precipIntensityError": 0.89,
  "precipType": "rain"
},

What the data section shows on the live API

{
  "time": 1712176380,
  "precipIntensity": 2.1094,
  "precipProbability": 0.91,
  "precipIntensityError": 0.86,
  "precipType": "rain"
},

Notice how on live it doesn't show "0":{

  • [x] Daily precipitationAccumulation in metric units is wrong unless only snow is forecast. ** This one was interesting, and your example from Remaining Bugs/Issues for the V2 Development API #180 was perfect to understand, what was happening, and why separate fields make more sense. The API picks a daily precipitation type based on the most common, and applies that factor. So if it's going to rain 4 hours for a total of 1 cm, and 12 hours of snow for a total of 10 cm, it'll show 20 cm. But I agree that it would make more sense to use the total, so switching it to that.

I thought on live it was a total and always in cm when using metric units. With the new fields precipAccumulation is always in cm like on the live API and liquid is in mm, snow is in cm, and ice is in mm?

cloneofghosts commented 8 months ago

@alexander0042 Did you forget to release v2.0h or are you still testing/fixing things?

alexander0042 commented 8 months ago

Took a bit longer to get to get those last couple points knocked off the list, and I wanted to validate some UV points as well. Pushed it live now though, so you should start seeing "h" it in about 5 minutes or so!

cloneofghosts commented 8 months ago

Yup, I see version 'h' now. I think the main bugs have been fixed but will test over the weekend to make sure nothing else funky is going on.

I closed the other issue I created since you can either look at the milestone or the project to see any remaining issues.

EDIT: Just to clarify the currently and hourly sections should show a precipitation icon if intensity is 0.1mm/h or greater (daily and hourly need to have a probability of 30% and greater)? It still seems to be set to 0,25mm/h.

alexander0042 commented 8 months ago

Great clarification question. As it stands now (and I don't know if I like this approach):

Currently

Hourly

Daily

This feels intuitively more or less correct, and I do like having different thresholds for snow or rain, since 1 mm is a lot of rain, but not a lot of snow. Maybe 0.25 mm/h rain+ 1 mm/h of snow would make more sense?

cloneofghosts commented 8 months ago

I'm not sure how other APIs handle the situation but I think that most show a precipitation icon if there's any precipitation currently or forecasted. As far as I can tell I think that's how Dark Sky did it as their translation module lists "very-light-X" as 0–0.4 mm/hr (they also say snow is about 1/3 of rain). You could always have the summary say something like "Possible Rain" for really low intensities.

For snowfall is the intensity converted from cm/h to mm/h in the precipIntensity? From my experience when the precipitation type changes from snow to rain or sleet I notice no difference. From your suggestion it seems like it would be harder for the snow icon to show compared to rain/sleet. My opinion would be the lower the threshold the better.

EDIT: Sometimes I'm getting version 2.0h and sometimes 2.0i and it seems to be random which one I get. Any idea on why this would be happening?

EDIT2: The subhourly HRRR doesn't have visibility does it? I've noticed times when the NBM visibility seems off compared to observed visibility at the airport. Now is a good example where with NBM enabled visibility is ~8km whereas HRRR is ~16km which is closer to the observed visibility of 24km at the airport.

alexander0042 commented 7 months ago

Great idea checking the translation library! I agree with the logic here, so I'll set currently to 0.02 mm/h, of rain or snow which is basically anything.

Looking at the project board, I think all the V2.0 requirements are in, since everything else isn't really part of spec to release it live! I'm thinking of keeping it isolated to the dev endpoint for another week or so, but then slowly rolling it out to the api one

cloneofghosts commented 7 months ago

The change to 0.02 mm/h makes sense to me. I think precipAccumulation is still using mm instead of cm as when I turn on version2 I see that precipAccumulation has the same data as liquidAccumulation. I thought that liquid and ice would be in mm but maybe everything is in cm?

"precipAccumulation": 1.6274,
"liquidAccumulation": 1.6274,
"snowAccumulation": 0,
"iceAccumulation": 0

It's too bad there is no visibility in the subhourly HRRR since it's the most up to date model and it would align better with the precipitation. Even though the HRRR data is a couple of hours delayed compared to the subhourly data it might still be worth it to switch to HRRR.

Thought of this the other day but you're not integrating NBM in Alaska (AK), Puerto Rico (PR), Hawaii (HI), Guam (GU) are you? Doesn't have to be for V2 but might be worth adding for those areas. There's also HRRR Alaska and HRRR Hawaii but they only update every 3h and may not be as worthwhile to add.

As for the remaining list I added a comment on the fire index task as there is data available in NBM though it's not global. The only other thing on the list was the self-hosting task which is just making the code available in a repo somewhere and adding some documentation on how to run the code.

alexander0042 commented 7 months ago

Good catch on the NBM fire index! I didn't see that, but it is exactly what I was looking for

cloneofghosts commented 7 months ago

The Fire Index is available for all NBM regions except for Guam and Oceanic. You'll still want to find a global model/source somewhere as NBM isn't available globally but it's a start.

Did you see my note above about precipAccumulation and visibility? Just making sure it wasn't missed. Also a few more things I noticed:

{
  "time": 1712840400,
  "icon": "cloudy",
  "summary": "Cloudy",
  "precipIntensity": 0,
  "precipProbability": 0,
  "precipIntensityError": -0.0401,
  "precipAccumulation": 0,
  "precipType": "rain",
  "temperature": 9.79,
  "apparentTemperature": 7.85,
  "dewPoint": 6.73,
  "humidity": 0.81,
  "pressure": 1003.2,
  "windSpeed": 12.96,
  "windGust": 18.72,
  "windBearing": 80,
  "cloudCover": 0.81,
  "uvIndex": 1.36,
  "visibility": 16.09,
  "ozone": 373.24
},
alexander0042 commented 7 months ago

The fire index was trickier than it should have been! It only comes out every 6 hours, so had to put it in as a new model (NBM_Fire). You might notice the run time is slightly off, but that's just a quirk for the next 3 hours, and should stabilize by tomorrow. In other 'k' news:

One other nice part of this change was that it showed the new (more flexible) approach to adding models works! Which will make it much easier to incorporate things in the future... maybe even let people submit PR's to add additional ones!

cloneofghosts commented 7 months ago

I'm getting the same issue as before where I get a mix of j and k versions. Couple of quick things I noticed about the fire index:

A couple of quick quesrions:

alexander0042 commented 7 months ago

Good idea about the currently, I forgot to add it there, so I'll put it in, along with a max time field. To answer those questions:

cloneofghosts commented 7 months ago

Yeah, I think it would make sense to have liquid and probably ice accumulation in mm as it makes sense for those precipitation types.

I can't really think of any other bugs so I think we're good for release. Just need to update the documentation (#185) for the 2.0 changes and the updated roadmap.

EDIT: I know this isn't your fault and there is likely not much you can do. The API tends to forecast many hours in a row of 57% cloud cover. Currently it's showing 57% cloud cover for now until 6pm.

cloneofghosts commented 7 months ago
  • No fire index data point on the currently section
  • There is no fireIndexMaxTime field on the daily section

@alexander0042 has this been added to the latest development version? I checked this morning and I didn't see it yet.

Yeah, I think it would make sense to have liquid and probably ice accumulation in mm as it makes sense for those precipitation types.

Haven't heard anything on this but are we going to leave it as is or change those to to mm instead of cm.

It's too bad there is no visibility in the subhourly HRRR since it's the most up to date model and it would align better with the precipitation. Even though the HRRR data is a couple of hours delayed compared to the subhourly data it might still be worth it to switch to HRRR.

Again, are we going to leave things as is or change to HRRR. I don't mind either way but using HRRR would sync things up better with the subhourly precipitation.

Thought of this the other day but you're not integrating NBM in Alaska (AK), Puerto Rico (PR), Hawaii (HI), Guam (GU) are you? Doesn't have to be for V2 but might be worth adding for those areas. There's also HRRR Alaska and HRRR Hawaii but they only update every 3h and may not be as worthwhile to add.

I think this was just missed. I checked yesterday and it doesn't look like you're using NBM in those regions. Would definitely improve accuracy as those regions are only using GFS/GEFS which update every 6h.

alexander0042 commented 7 months ago

Should see version "l" coming online shortly with these fixes! In particular:

And to answer those questions:

cloneofghosts commented 7 months ago

I haven't encountered any more issues so I think it's ready for release. The V2 changes will need to be documented but there's another issue for that. I will keep this open until release in case anyone else encounters any more issues.

trevorturk commented 7 months ago

(Just wanted to chime in to say that I'm so excited for v2, thank you for all of your hard work!)

alexander0042 commented 7 months ago

Thanks for saying that @trevorturk! It was your discussion/ research into the daily max/,min that started me down this (very slow road), so means a lot to hear that

cloneofghosts commented 7 months ago

@alexander0042 Just noticed a few last minute bugs with the development version:

cloudCover can go below 0 on hourly:

{
  "time": 1713600000,
  "icon": "clear-night",
  "summary": "Clear",
  "cloudCover": -0.03,
},

I also noticed the bug with longitude for nbm and hrrr in the sourceIDX block that happened with the location longitude where -75 is showing as 285.

cloneofghosts commented 7 months ago

I also saw that precipAccumulation had more than 4 decimal places yesterday but I do not see the same issue this morning. Might be when there are more than one type of precipitation for the day?

alexander0042 commented 7 months ago

Great final catches, and put them in! Today's the official go live day, so I'm going to (very slowly) start moving the production endpoint over to V2.0.1. Definitely let me know if anything crops up, and since V1 will keep running in the background as a fallback

cloneofghosts commented 7 months ago

The version number in the flags section seems to have disappeared. It may have happened when you removed the processTime from the flags section earlier.

cloneofghosts commented 7 months ago

It's been a few days and everything seems stable so I'll close this issue. Thanks to everyone who tested the development version.