Preprocess/ndvi atzberger

[tommylees112]

Download and preprocess the NDVI Data from BOKU (the operational data!!!!!)

NOTE: This has expanded

• added a sortby() to the DataLoader class (because of the inversion of the ADede only data)
• a load of notebooks
• adede variable experiments
• Boku NDVI preprocessing and Exporting
• Return a Path object from the Preprocessor.merge_files() argument and therefore some changes to the ERA5Preprocessor for mypy to be happy
• FIX THE CONDITION INDEX (needs testing)
• Using DEKADS as a valid timestamp / time resampler

Woops! (But panicked prep for ICLR)

TODO:

• update the unit tests for the BokuNDVIExporter

 Done

• before merge we need to find the bug that is causing the latitudes to switch in some of the variables giving us the UPSIDE DOWN predictions.
• it has to be relatively new (since writing the 11_.py experiment script) because it wasn't a problem for models trained before that.
• write a unit test to catch that shit before it happens again and some assert statements once we identify the problem
• update the unit tests for the Condition Index
• update the unit tests for the BokuNDVIPreprocessor

 where?

• src/models/neural_networks/base.py -> in train()
• src/models/base.py -> in evaluate()