search

blaylockbk / Herbie

Download numerical weather prediction datasets (HRRR, RAP, GFS, IFS, etc.) from NOMADS, NODD partners (Amazon, Google, Microsoft), ECMWF open data, and the University of Utah Pando Archive System.
https://herbie.readthedocs.io/
MIT License
497 stars 74 forks source link

Add Canadian models #206

Open blaylockbk opened 1 year ago

blaylockbk commented 1 year ago

Requested by a user in an email

Add model templates for Candadian models

Data source - Meteorological Service of Canada DataMart

Note: Canada has a lot of NetCDF; supporting this would require significant Herbie refactoring.

blaylockbk commented 1 year ago

Start with HRDPS https://weather.gc.ca/grib/grib2_HRDPS_HR_e.html

bryanguarente commented 1 year ago

Brian, Glad you are trying this out. I tried but failed pretty miserably. There are no grib index files so your methodology seems to fail this dataset. I am sure we could build some grib.idx files to help out, but for now I haven't yet had time. Let me know if you need some assistance on this.

As for your question of what are these models I know a few of them:

I hope that helps you in some way.
Bryan

blaylockbk commented 1 year ago

Thanks @bryanguarente for the description. That is helpful. Someone from ECCC asked if it could be done so I wanted to try. The lack of index files is a problem; Herbie would only be able to download full files and subset locally.

In my poking around, the HRDPS looks like there is only one grib message per file?? I'll have to look into it more to understand the files; I've never used these before.

bryanguarente commented 1 year ago

@blaylockbk yeah, they only put one variable in each file. It is an interesting methodology, but would be super fast to get to a single output.

I have a list of the variables available in GDPS (gem_global) if that would be useful for you.

blaylockbk commented 1 year ago

Yes, that would be nice.

bryanguarente commented 1 year ago

All variables available for GDPS at f006: "ABSVIBSL[0-9][0-9][0-9]" --> Absolute Vorticity on Isobaric surface [1015, 1000, 985, 975, 950, 925, 900, 875, 850, 800, 750, 700, 650, 600, 550, 500, 450, 400, 350, 300, 275, 250, 225, 200, 175, 150, 125, 100, 50, 30, 20, 10] "ALBDO_SFC_0" --> Albedo at the surface "ACPCP_SFC_0" --> 6hr accumulation of convective precipitation "APCP_SFC_0" --> 6hr accumulation of precipitation "CAPE_SFC_0" --> Surface-based CAPE "CIN_SFC_0" --> Surface-based CIN "CWAT_EATM_0" --> Cloud Water (entire depth of atmosphere) "DEPR_TGL_2" --> Dewpoint depression (2m above ground) "DEPRISBL[0-9][0-9][0-9]" --> Dewpoint depression on Isobaric surface [1015, 1000, 985, 975, 950, 925, 900, 875, 850, 800, 750, 700, 650, 600, 550, 500, 450, 400, 350, 300, 275, 250, 225, 200, 175, 150, 125, 100, 50, 30, 20, 10] "DLWRF_SFC_0" --> Downward longwave radiation at the surface "DSWRF_NTAT" --> Downward shortwave radiation near the top of the atmosphere "DSWRF_SFC_0" --> Downward shortwave radiation at the surface "DPT_TGL_2" --> Dewpoint temperature at 2m above ground "GUST_MAX_TGL_10" --> Maximum (0-hour) wind gust at 10 m above ground "GUST_MIN_TGL_10" --> Minimum (0-hour) wind gust at 10 m above ground "GUST_TGL_10" --> Wind gust at 10 m above ground "HGTISBL[0-9][0-9][0-9]" --> Geopotential Height on Isobaric surface [1015, 1000, 985, 975, 950, 925, 900, 875, 850, 800, 750, 700, 650, 600, 550, 500, 450, 400, 350, 300, 275, 250, 225, 200, 175, 150, 125, 100, 50, 30, 20, 10, 1] "HGT_ISBY_1000-500" --> Thickness 1000hPa to 500hPa "HPBL_SFC_0" --> Planetary Boundary Layer Height "KX_SFC_0" --> K-Index "LHTFL_SFC_0" --> Latent Heat Flux at the Surface "MU-VT-LI_ISBL_500" --> 4-Layer Best Lifted Index at 500hPa "NLWRS_SFC_0" --> Net Longwave Radiation at the surface (6 hour accumulation) "NSWRS_SFC_0" --> Net Shortwave Radiation at the surface (6 hour accumulation) "PRATE_SFC_0" --> Precipitation Rate at the surface "PRMSL_MSL_0" --> Pressure Reduced to Mean Sea Level at the surface "PRES_PVU_1" --> Pressure of the 1.0 PVU surface "PRES_PVU_1.5" --> Pressure of the 1.5 PVU surface "PRES_PVU_2" --> Pressure of the 2.0 PVU surface "PRES_SFC_0" --> Pressure at the surface "PTYPE_SFC_0" --> Precipitation type at the surface "RH_TGL_2" --> Relative Humidity at 2m above ground "RHISBL[0-9][0-9][0-9]" --> Relative Humidity on Isobaric surface [1015, 1000, 985, 975, 950, 925, 900, 875, 850, 800, 750, 700, 650, 600, 550, 500, 450, 400, 350, 300, 275, 250, 225, 200, 175, 150, 125, 100, 50, 30, 20, 10] "SDEN_SFC_0" --> Snow density at the surface "SFCWRO_SFC_0" --> Surface water run-off "SHTFL_SFC_0" --> Sensible Heat Flux at the surface "SHWINX_ISBL_500" --> Showalter Index at the 500hPa surface "SKINT_SFC_0" --> Skin Temperature "SNOD_SFC_0" --> Snow Depth at the surface "SPFH_SFC_0" --> Specific Humidity at the surface "SPFH_TGL_2" --> Specific Humidity at 2 m above ground "SPFH_TGL_40" --> Specific Humidity at 40 m above ground "SPFH_TGL_80" --> Specific Humidity at 80 m above ground "SPFH_TGL_120" --> Specific Humidity at 120 m above ground "SPFHISBL[0-9][0-9][0-9]" --> Specific Humidity on Isobaric Surface [1015, 1000, 985, 975, 950, 925, 900, 875, 850, 800, 750, 700, 650, 600, 550, 500, 450, 400, 350, 300, 275, 250, 225, 200, 175, 150, 125, 100, 50, 30, 20, 10, 1] "SOILVIC_SFC_0" --> Soil Volumetric Ice Content Water Equivalent at the surface "SOILW_DBLL_100" --> Soil moisture content at 100cm below surface "SOILW_DBLY_10" --> Soil moisture content at 10cm below surface "TCDC_SFC_0" --> Total Cloud cover full column "TMAX_TGL_2" --> Max temperature at 2m above ground (last 6 hours) "TMIN_TGL_2" --> Max temperature at 2m above ground (last 6 hours) "TMP_SFC_0" --> Temperature at the surface "TMP_TGL_2" --> Temperature at 2m above ground "TMP_TGL_40" --> Temperature at 40m above ground "TMP_TGL_80" --> Temperature at 80m above ground "TMP_TGL_120" --> Temperature at 120m above ground "TMPISBL[0-9][0-9][0-9]" --> Temperature on Isobaric Surface [1015, 1000, 985, 975, 950, 925, 900, 875, 850, 800, 750, 700, 650, 600, 550, 500, 450, 400, 350, 300, 275, 250, 225, 200, 175, 150, 125, 100, 50, 30, 20, 10, 1] "TOTALX_SFC_0" --> Total totals at the surface "TSOIL_DBLL_10cm" --> Soil Temperature at 10cm below ground "TSOIL_SFC_0" --> Soil Temperature at the surface "UGRD_TGL_10" --> U-component of the wind at 10m above ground "UGRD_TGL_40" --> U-component of the wind at 40m above ground "UGRD_TGL_80" --> U-component of the wind at 80m above ground "UGRD_TGL_120" --> U-component of the wind at 120m above ground "UGRDISBL[0-9][0-9][0-9]" --> U-component of the wind on Isobaric Surface [1015, 1000, 985, 975, 950, 925, 900, 875, 850, 800, 750, 700, 650, 600, 550, 500, 450, 400, 350, 300, 275, 250, 225, 200, 175, 150, 125, 100, 50, 30, 20, 10, 1] "ULWRF_NTAT_0" --> Upward Longwave Radiation near the top of the atmosphere "USWRF_SFC_0" --> Upward Shortwave Radiation at the surface "VGRD_TGL_10" --> V-component of the wind at 10m above ground "VGRD_TGL_40" --> V-component of the wind at 40m above ground "VGRD_TGL_80" --> V-component of the wind at 80m above ground "VGRD_TGL_120" --> V-component of the wind at 120m above ground "VGRDISBL[0-9][0-9][0-9]" --> V-component of the wind on Isobaric Surface [1015, 1000, 985, 975, 950, 925, 900, 875, 850, 800, 750, 700, 650, 600, 550, 500, 450, 400, 350, 300, 275, 250, 225, 200, 175, 150, 125, 100, 50, 30, 20, 10, 1] "VVELISBL[0-9][0-9][0-9]" --> Vertical Velocity on Isobaric Surface [1015, 1000, 985, 975, 950, 925, 900, 875, 850, 800, 750, 700, 650, 600, 550, 500, 450, 400, 350, 300, 275, 250, 225, 200, 175, 150, 125, 100, 50, 30, 20, 10, 1] "WDIR_TGL_10" --> Wind direction at 10m above ground "WDIR_TGL_40" --> Wind direction at 40m above ground "WDIR_TGL_80" --> Wind direction at 80m above ground "WDIR_TGL_120" --> Wind direction at 120m above ground "WDIRISBL[0-9][0-9][0-9]" --> Wind direction on Isobaric Surface [1015, 1000, 985, 975, 950, 925, 900, 875, 850, 800, 750, 700, 650, 600, 550, 500, 450, 400, 350, 300, 275, 250, 225, 200, 175, 150, 125, 100, 50, 30, 20, 10, 1] "WEAFR_SFC_0" --> Water equivalent accumulated freezing rain at the surface "WEAPE_SFC_0" --> Water equivalent accumulated ice pellets at the surface "WEARA_SFC_0" --> Water equivalent accumulated rain at the surface "WEASN_SFC_0" --> Water equivalent accumulated snow at the surface "WIND_TGL_10" --> Wind speed at 10m above ground "WIND_TGL_40" --> Wind speed at 40m above ground "WIND_TGL_80" --> Wind speed at 80m above ground "WIND_TGL_120" --> Wind speed at 120m above ground "WINDISBL[0-9][0-9][0-9]" --> Wind speed on Isobaric Surface [1015, 1000, 985, 975, 950, 925, 900, 875, 850, 800, 750, 700, 650, 600, 550, 500, 450, 400, 350, 300, 275, 250, 225, 200, 175, 150, 125, 100, 50, 30, 20, 10, 1]