google-research / weatherbench2

A benchmark for the next generation of data-driven global weather models.
https://weatherbench2.readthedocs.io
Apache License 2.0
402 stars 40 forks source link

Feature request: Evaluating at finer temporal resolution #95

Open erinboyle opened 9 months ago

erinboyle commented 9 months ago

I'm probably not the only person out there that is excited to use GraphCast but limited by the 6 hour resolution. I work in the energy sector. Forecasting the shape of electricity prices across the day matters a lot to battery revenue, and is highly dependent on weather. Accuracy at ~hourly resolution probably determines when generic weather apps can switch to new models, too.

So the request is to track how models perform at hourly resolution, instead (or in addition). (I'm guessing models capable of this, or even just postprocessing that competes, will come out soon). Thanks for considering!

The-Fonz commented 9 months ago

I second this, would be very useful for the electricity forecasting industry. Also see my request to evaluate on a finer spatial grid.

raspstephan commented 9 months ago

Thanks for the comment. This is a very good point and it really helps to hear what people are interested in. I assume you'd be interested in this mostly for the first 24h? It will require some work to download and process the baselines in hourly resolution. In addition, current ML models (As you mention) are 6 hourly at best but finer resolution will definitely come, at which point we will want to include it in our benchmark. So this is definitely on the roadmap but I wouldn't expect something in the immediate future, sorry ;)

erinboyle commented 8 months ago

I assume you'd be interested in this mostly for the first 24h?

Actually, 48 hours would be great if possible / if you want to fully support the electricity use-case. :) We are generally forecasting midnight-midnight horizons at 9 am the previous day, because that's when the "day-ahead" market bids are due.

raspstephan commented 8 months ago

Thanks for the info!