search

openclimatefix / graph_weather

PyTorch implementation of Ryan Keisler's 2022 "Forecasting Global Weather with Graph Neural Networks" paper (https://arxiv.org/abs/2202.07575)
MIT License
180 stars 45 forks source link

Train on GFS Data #4

Open jacobbieker opened 2 years ago

jacobbieker commented 2 years ago

Detailed Description

GFS data is publicly accessible and so we can make that dataset open, as well as its one of the ones used in the original paper, although with 1 degree data, not the 0.25 degree data I want to try with.

Context

Possible Implementation

jacobbieker commented 2 years ago

Data is being pushed to here for training: https://huggingface.co/datasets/openclimatefix/gfs-reforecast which contains/will conatin raw observations, processed observations, GFS operational forecasts, and FNL Analysis files on 0.25 degree grids for 2016-2022. As well as the GEFS reforecasts from 2010-2019, and potentially back to 2000. Everything is in Zarr format, as it compressed a lot better, although it does mean that the data files all need to be decompressed on download.

jacobbieker commented 2 years ago

Also, just got a model training on 0.25 degree analysis file data for 2016-01-01. It learns things and fits on an 8GB 2070 laptop GPU if downsampling to 1 degree resolution.