openclimatefix / nowcasting_dataset

Prepare batches of data for training machine learning solar electricity nowcasting data
https://nowcasting-dataset.readthedocs.io/en/stable/
MIT License
24 stars 6 forks source link

Add weather observations #309

Open JackKelly opened 2 years ago

JackKelly commented 2 years ago

Detailed Description

Add weather observation data such as temperature, wind speed & direction, rainfall, humidity, and irradiance from sources such as:

Some data sources are listed here: https://docs.google.com/document/d/1tk9cF4O539TzaMaUufn9Ay4f6qKKEyoNKmzP03kbSDo/edit#

Context

Ultimately, we're trying to teach ML models to forecast the weather better than numerical weather models (over time horizons of a few hours). To do that, our ML models need as much information as possible about the recent state of the atmosphere.

All of these weather observations are sparse in time and/or space. But that's fine! One of the awesome things about Perceiver IO is that it can handle sparse observations.

This definitely isn't something to prioritise for 2021, though!

jacobbieker commented 2 years ago

Somewhat related is the rainfall radar: #80

JackKelly commented 2 years ago

Good point! After your suggestion, I've added #80 to the tick-list above.