search

openclimatefix / graph_weather

PyTorch implementation of Ryan Keisler's 2022 "Forecasting Global Weather with Graph Neural Networks" paper (https://arxiv.org/abs/2202.07575)
MIT License
195 stars 50 forks source link

[Paper & Dataset] Verification against in-situ observations forData-Driven Weather Prediction #70

Open jacobbieker opened 1 year ago

jacobbieker commented 1 year ago

Arxiv/Blog/Paper Link

https://arxiv.org/pdf/2305.00048.pdf

Detailed Description

Dataset is available through Huggingface here: https://huggingface.co/datasets/excarta/madis2020 This paper goes more into verifying how well data driven model perform, specifically against real observations as well, compared to the common benchmark of ERA5 reanalysis. The data in the dataset is from MADIS.

Context

It would be good to compare what these models can do against observations.

byphilipp commented 1 year ago

I see a very strange results in this paper: the errors is not increase respect to lead time The good idea is using the NOAA-ISD dataset for this verification - it is globally and contains the quality control checks https://www.ncei.noaa.gov/data/global-hourly/

ch1booze commented 7 months ago

If this issue is still open, I would like to take it on. I have perused the paper and what I understand is that the methods evaluation of DDWPs tests how well the model can replicate the data it has ingested. However, the real world has unforeseen circumstances and thus, what is required here is to evaluate DDWPs with real-world scenarios.

jacobbieker commented 7 months ago

Yes, it is! And yeah, happy to have you take it!