Windnet DA model not working as it should

[peterdudfield]

DA windnet forecast, is not better. The version on dev, should be better that production from ML experiments. Production has a smaller patch size than dev

https://github.com/openclimatefix/PVNet/blob/main/experiments/india/003_wind_plevels/plevel.md

It feels like there is something wrong after 35 hours. And slightly strange that around 24 hours it is worst that production.

For example the 2700 forecast horizon, the results are all too high

Ideas:

• [ ] New batches that the model was trained on, dont represent the data properly. There was a weird init time bug where there were multiple init times
• [ ] Live batches contain nans
• [ ] Are the training batches and live batches normalised in the same way.
• [x] Batches and live, are they using the same version of ocf_datapipes?

Batch creation was made using ocf_datapipes==3.3.24. Current version in this repo isocf-datapipes = "3.3.7", so we could try upgrading this

• [x] Do training batches have a good distribution of init times

Looked at the init values in the train and validation, and looks good

[peterdudfield]

Looking also at retraining a model with a different seed, perhaps.3 different models, to see the differences in results