ai2cm / fv3net

explore the FV3 data for parameterization
MIT License
16 stars 3 forks source link

Time shift hybrid data during preprocessing #2333

Closed AnnaKwa closed 12 months ago

AnnaKwa commented 1 year ago

The hybrid reservoir model takes in the state at time t, updates its hidden state, and then uses its hidden state plus the 'imperfect' (i.e. coarse model FV3GFS) prediction at time t+1 to predict the corrected state at time t+1. We previously discussed how the hybrid inputs should be shifted by an index -1 to match up with the hidden states at time t. However, I did not merge the agreed upon solution (do the shifting in the preprocessing) so my latest results from hybrid models have the incorrect time index of hybrid inputs.

This PR corrects the issue by shifting hybrid inputs in the preprocessing step.

It also includes a fix to only reset the reservoir state at the start of each initial condition's time series. Previously it was resetting every batch. Previously trained models used a branch with this fix so they weren't affected.