SolarArbiter / solarforecastarbiter-core

Core data gathering, validation, processing, and reporting package for the Solar Forecast Arbiter
https://solarforecastarbiter-core.readthedocs.io
MIT License
33 stars 21 forks source link

Test against upgraded GEFS #544

Open wholmgren opened 4 years ago

wholmgren commented 4 years ago

Implementation currently scheduled for late September.

Parallel data is available

https://www.emc.ncep.noaa.gov/users/meg/gefsv12/

wholmgren commented 4 years ago

New GEFS was implemented a few days and I'm somewhat surprised to say that we're still seeing new forecasts.

Things to consider changing:

Use 0.25 degree grid available at https://nomads.ncep.noaa.gov/cgi-bin/filter_gefs_atmos_0p25s.pl

https://github.com/SolarArbiter/solarforecastarbiter-core/blob/4ee531a4851f5d7bbdd598d2758acbf8a3abad09/solarforecastarbiter/io/fetch/nwp.py#L175-L176

Increase number of perturbed members from 20 to 30:

https://github.com/SolarArbiter/solarforecastarbiter-core/blob/4ee531a4851f5d7bbdd598d2758acbf8a3abad09/solarforecastarbiter/io/fetch/nwp.py#L195

3 hourly through 240 hours:

https://github.com/SolarArbiter/solarforecastarbiter-core/blob/4ee531a4851f5d7bbdd598d2758acbf8a3abad09/solarforecastarbiter/io/fetch/nwp.py#L188-L189

0.5 deg data is available in 6 hour intervals through 840 hours on the 0Z cycle, 384 hours on the 6Z/12Z/18Z cycles.

Add a new function gefs_quarter_deg_to_hourly_mean similar to

https://github.com/SolarArbiter/solarforecastarbiter-core/blob/4ee531a4851f5d7bbdd598d2758acbf8a3abad09/solarforecastarbiter/reference_forecasts/models.py#L499-L593

Modify the 3/6 hour switch in the unmixing function

https://github.com/SolarArbiter/solarforecastarbiter-core/blob/4ee531a4851f5d7bbdd598d2758acbf8a3abad09/solarforecastarbiter/reference_forecasts/models.py#L596-L599

Assuming we want to keep the [0, 5...95, 100] GEFS forecast percentiles, we need to map the 31 members onto 21 percentiles. That probably requires inserting logic in run_nwp after these lines:

https://github.com/SolarArbiter/solarforecastarbiter-core/blob/4ee531a4851f5d7bbdd598d2758acbf8a3abad09/solarforecastarbiter/reference_forecasts/main.py#L143-L144

alorenzo175 commented 4 years ago

New forecasts are not being fetched. Looks like the directory format has changed, so the fetch script chokes.

alorenzo175 commented 3 years ago

Thinks not addressed by #584: