[x] restructure output netcdf files for intra-annual metrics
[x] implement compression for output netcdf files
[ ] adapt plotting accordingly
new files:
validator.validation.intra_annual_slicer.py: contains logic to create intra-annual slices a.k.a. temporal sub-windows
validator.validation.netcdf_transcription.py: contains logic to translate the original pytesmo output netCDF file into the new qa4sm one
intra_annual_slices_definition.json: not required for default cases, those are defined in the globals. kept to try injecting custom sub-temporal windows for now (as this functionality is implemented in validator.validation.intra_annual_slicer.IntraAnnualSlicesDefault to add flexibility for users)
files containing new code {all new lines are commented with #$$}:
validator.validation.globals.py
validator.validation.validation.py:
new block at beginning (l54-61) handling intra-annual slice definitions
changes in save_validation_config()
introduced context manager to handle writing of netcdf.Dataset
added qa4sm_reader_version to metadata (doesn't the environment.yaml file need to be adapted too when publishing the branch?)
takes second argument now, being a validator.validation.netcdf_transcription.Pytesmo2Qa4smResultsTranscriber instance
changes in create_pytesmo_validation():
l 371-391
changes in run_validation():
l 623-642
validator.tests.test_validation.py:
adapted check_results(),
takes additional argument now, being a validator.validation.netcdf_transcription.Pytesmo2Qa4smResultsTranscriber instance. Accordingly, all occurrences of this method throughout this file were adapted
l 208f
l 269f: some data variables (=metrics) now are a function of a new coordinate, being the temporal-sub window. Thus, the shape of these metrics changed and has to be accounted for when testing their lengths
l 377-432: ignored the plotting tests, as now plotting is done currently
adapted all 20 occurrences of check_results() and added a code block prior to it, instantiating a transcriber instance. Note: currently the tests are only adapted to run for the bulk scenario. I did not add the #$$ comment to all 20 occurrences, though....
progress so far:
new files:
pytesmooutput netCDF file into the newqa4smoneglobals. kept to try injecting custom sub-temporal windows for now (as this functionality is implemented invalidator.validation.intra_annual_slicer.IntraAnnualSlicesDefaultto add flexibility for users)files containing new code {all new lines are commented with #$$}:
save_validation_config()netcdf.Datasetqa4sm_reader_versionto metadata (doesn't the environment.yaml file need to be adapted too when publishing the branch?)validator.validation.netcdf_transcription.Pytesmo2Qa4smResultsTranscriberinstancecreate_pytesmo_validation():run_validation():check_results(),validator.validation.netcdf_transcription.Pytesmo2Qa4smResultsTranscriberinstance. Accordingly, all occurrences of this method throughout this file were adaptedcheck_results()and added a code block prior to it, instantiating a transcriber instance. Note: currently the tests are only adapted to run for the bulk scenario. I did not add the #$$ comment to all 20 occurrences, though....test_errors(): l 1560-1566