Include GFS data from NCEI and/or RDA

blaylockbk / Herbie

Download numerical weather prediction datasets (HRRR, RAP, GFS, IFS, etc.) from NOMADS, NODD partners (Amazon, Google, Microsoft), ECMWF open data, and the University of Utah Pando Archive System.

https://herbie.readthedocs.io/

MIT License

480 stars 72 forks source link

Include GFS data from NCEI and/or RDA #325

Closed blaylockbk closed 1 month ago

blaylockbk commented 3 months ago

https://www.ncei.noaa.gov/products/weather-climate-models/global-forecast

raulv48 commented 3 months ago

I just tweak Herbie today to work with NCEI GFS. This will be a welcomed addition.

vwgeiser commented 2 months ago

@raulv48 With this are you able to access GFS runs prior to 2021?

blaylockbk commented 1 month ago

GFS is also archived on RDA, but these files don't have .idx files with them which makes variable subsetting impossible.

https://rda.ucar.edu/datasets/d084001/

NOTE: downloads are very slow from RDA.

blaylockbk commented 1 month ago

GFS on NCEI via HTTPS server is spotty at best

Example: https://www.ncei.noaa.gov/data/global-forecast-system/access/grid-003-1.0-degree/analysis/

blaylockbk commented 1 month ago

Closed for these reasons

GFS files at NCEI via HTTPS are spotty. Not sure it's worth making a Herbie template for this.
GFS file access from NCAR RDA was implemented in #358, but given there is no inventory file, the user has no choice but to download the full file. You could generate the inventory for the file with wgrib2 after the file is downloaded as described here.

raulv48 commented 1 month ago

@raulv48 With this are you able to access GFS runs prior to 2021?

@vwgeiser Yes, I was able to download NCEI data from 2020.

Sorry for the late comment on a closed ticket. Just wanted to answer as it is possible by updating the NCEI template.

blaylockbk commented 1 month ago

Opening this again because I see what you mean about being able to get 2020 GFS data

I can see a fair amount of analysis data available from May-Dec 2020. There isn't much available before those dates.

I'll update the template.