google-deepmind / graphcast

Apache License 2.0
4.36k stars 537 forks source link

extract_input_target_forcings add option for left-justification of train/eval #56

Open tvonich opened 5 months ago

tvonich commented 5 months ago

Provides optionality for a left-justified evaluation and training with kwarg justify = 'left' or 'right'. Changes are from lines 281 - 322, line 361, and line 381.

Per issue discussion with Timo Ewalds #32

justify = 'left' is now the default and I believe that makes sense based on normal user expectation of left to right progression of time when doing a forecast and selecting data.

To explain what is different now, I'll use the following example.

We load dataset_source-era5_date-2022-01-01_res-1.0_levels-13_steps-12.nc in the Jupyter Notebook.

There are 14 time periods along the data dimension for this netcdf: 2 initial states + 12 steps

Let's visualize them as: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14

We set Eval Steps and Train Steps to 4 on the accompanying slider and extract the data with extract_input_target_forcings.

With the previous extract_input_target_forcings function, the inputs would've been 9, 10 and the targets would've been 11, 12, 13, 14 for the designated Eval and Train Steps. This is a bit misleading because if I want to do a 24-hour forecast starting at 0Z 01-01-2022, I expect my forecast end time to be 0Z 01-02-2022. In the right justified method, the forecast always ends on the last time element within the example_data.

With the new left-justified default, the inputs are always the 1 and 2, with the targets following immediately after. So in the above example, that would be 1, 2 for inputs and 3, 4, 5, 6 for targets.

tvonich commented 5 months ago

Tried to accomplish the CLA...may be doing it wrong. If any clarification is available that'd be great.

tvonich commented 4 months ago

I'll work through these in the next week or two when able. And yes, I can show that it's working as expected.