search

jweyn / DLWP-CS

Deep learning models for global weather prediction on a cubed sphere
GNU General Public License v3.0
90 stars 40 forks source link

Validation data and Forecast data error #7

Open priya-sharma-07 opened 3 years ago

priya-sharma-07 commented 3 years ago

@jweyn could you please suggest a file that actually converts the validation data in the format of forecast data. because in forecast data the format is (f_hour, sample, lat, lon, varlev) and in validation data the format is (time, lat, lon, varlev). actually, we are testing the model on the ERA5 dataset subregion(2D data). we are able to train the model with a loss of 0.0234. but for prediction, we are not getting expected results. Maybe we are not able to compare validation data and forecast data correctly. it would be great if you could suggest a solution.

training1 prediction1

ashish615 commented 3 years ago

Even I am facing the same issue while calculating error i am not able to reproduce results as per reported. I am forecasting for the 7 days and 6h interval. Histogram represents the number of instances with given rmse value(ranging from 1 to 15). Please Help me with this.

Screenshot from 2021-03-18 09-41-13

jweyn commented 3 years ago

Hi @priya-sharma-07, to convert the validation data to the same format as the model prediction, first understand that the "sample" dimension corresponds to the time of forecast initialization while the "f_hour" dimension corresponds to the number of forecast hours in the future relative to the initialization time. So for example, sample=2018-01-01 and f_hour=108 means the validation would be at time=2018-01-04 12:00. There is a method that converts training data to a forecast-like array in DLWP.verify.verification_from_series.

Note that this conversion will result in some NaNs for the latest initialization times at the end of a time series, since the forecast will be for some future time that hasn't occurred (isn't in the original data).

jweyn commented 3 years ago

@ashish615 , I'm not sure I understand what is in your figure or what your issue is. Does the above comment help?