search

openclimatefix / graph_weather

PyTorch implementation of Ryan Keisler's 2022 "Forecasting Global Weather with Graph Neural Networks" paper (https://arxiv.org/abs/2202.07575)
MIT License
187 stars 47 forks source link

[Paper] GenCast Diffusion model for weather forecasting #80

Open jacobbieker opened 9 months ago

jacobbieker commented 9 months ago

Arxiv/Blog/Paper Link

https://arxiv.org/abs/2312.15796

Detailed Description

A diffusion-based approach to weather forecasting, that is quite stable autoregressively. Probably fits in more with diffusion_weather but here to be with the other weather papers. They also say the model is based on GraphCast, but with a different graph connectivity, and a sparse transformer instead of the Processor GNN in GraphCast.

Context

Cool way of doing ensemble predictions, scalable, only at a 1 degree resolution though, 12 hour timesteps as well, so less than Graphcast and the like for some reason?

jacobbieker commented 9 months ago

The encoder and decoder are the same as GraphCast, but the latent grid is a 5-refined mesh, not a multi-mesh, with 10242 nodes and 61440 edges.

image

jacobbieker commented 9 months ago

I think this adds more support to modularizing graph_weather, what is being done in #76, so that it is easier to experiment with/replicate this kind of result

jacobbieker commented 9 months ago

They train on the 12 hour timestep to be in different data assimilation windows, as ERA5 only has 2 a day.

jacobbieker commented 9 months ago

Overall, really impressive results I think. Interesting combination of graph and diffusion model. A lot slower to run and train because of how diffusion models work, and still requires NWP analysis field for initialization. Compared to a few seconds for a 0.25 degree forecast with GraphCast, 1min per forecast with GenCast at 1 degree is a lot slower. But the better results, and it still be a lot faster than traditional methods make it quite interesting.

jacobbieker commented 9 months ago

Would be really keen to implement this here.

aavashsubedi commented 6 months ago

Seems like an interesting project. Is this open as a GSOC project? And if so, what would the scope/length be? (I don't see GraphCast in the repo, so I imagine that would also need to be ported over from DM ?)

jacobbieker commented 6 months ago

Yes, this could work as a GSoC project. It would be a large project (350h). GraphCast wouldn't need to be ported over, we already have the encoder/decoder graph networks implemented, although they definitely can be improved! It would be more the diffusion model that would need to be added. And some changes to make the code more modular, so that we can easily swap out Encoders/Processor/Decoders easily.

jacobbieker commented 4 months ago

They have released an updated paper now, increasing the resolution to 0.25 degrees, and more comparison against ENS, which they claim to beat in 97 percent of the time.

gbruno16 commented 4 months ago

This is very cool! Also, it seems that you don't need to restart training from scratch to go from 1 degree to 0.25: just a few small modifications and fine-tuning are enough, similar to graph weather!