The aim of this PR is to get the data download/pre-process script into the package, add more pre-processing that was being done elsewhere, and clean the code up.
The easiest way to see what's new is to look where the new linmod.data module is used:
exploration/demo/README.md describes how to run this part of the pipeline, which now no longer requires a path reference to the script
exploration/demo/*.py have shorter, cleaner code chunks for data = ..., because that processing was moved to the package
The major changes to linmod.data are:
Configuration parameters are global constants now
Data pre-processing will filter collection dates to be no later than the specified forecast date
The offset from the forecast date is included in the pre-processed data as fd_offset
The aim of this PR is to get the data download/pre-process script into the package, add more pre-processing that was being done elsewhere, and clean the code up.
The easiest way to see what's new is to look where the new
linmod.datamodule is used:exploration/demo/README.mddescribes how to run this part of the pipeline, which now no longer requires a path reference to the scriptexploration/demo/*.pyhave shorter, cleaner code chunks fordata = ..., because that processing was moved to the packageThe major changes to
linmod.dataare:fd_offsetSmaller changes are
linmod.evalhad a couple bugs that snuck in