[x] restructure output netcdf files for intra-annual metrics
[x] implement compression for output netcdf files
[ ] adapt plotting accordingly (undergoing in qa4sm_reader)
new files:
validator.validation.intra_annual_slicer.py: contains logic to create intra-annual slices a.k.a. temporal sub-windows
validator.validation.netcdf_transcription.py: contains logic to translate the original pytesmo output netCDF file into the new qa4sm one
intra_annual_slices_definition.json: not required for default cases, those are defined in the globals. kept to demonstrate injecting custom sub-temporal windows for now (as this functionality is implemented in validator.validation.intra_annual_slicer.IntraAnnualSlicesDefault to add flexibility for users)
files containing new code {new lines are commented with #$$}:
validator.validation.globals.py
validator.validation.validation.py:
new block at beginning (l54-65) handling intra-annual slice definitions
thats what would need to be defined in the front end in the future
changes in save_validation_config()
introduced context manager to handle writing of netcdf.Dataset
added qa4sm_reader_version to metadata (doesn't the environment.yaml file need to be adapted too when publishing the branch?)
takes second argument now, being a validator.validation.netcdf_transcription.Pytesmo2Qa4smResultsTranscriber instance
changes in create_pytesmo_validation():
l 380-397
changes in run_validation():
l 629-650
validator.tests.test_validation.py: Tests are NOT completely adapted yet
adapted check_results(),
takes additional argument now, being a validator.validation.netcdf_transcription.Pytesmo2Qa4smResultsTranscriber instance. Accordingly, all occurrences of this method throughout this file were adapted
adapted all 20 occurrences of check_results() and added a code block prior to it, instantiating a transcriber instance. Note: currently the tests are only adapted to run for the bulk scenario. I did not add the #$$ comment to all 20 occurrences, though....
adapted test_errors(): l 1560-1566
Outlook:
[ ] Adapt all test for default case
[ ] Add tests for temporal sub-window cases
[ ] Make sure tests handle new graphics output of qa4sm_reader
This is a draft and not meant for production!
things to be removed:
progress so far:
qa4sm_reader)new files:
pytesmooutput netCDF file into the newqa4smoneglobals. kept to demonstrate injecting custom sub-temporal windows for now (as this functionality is implemented invalidator.validation.intra_annual_slicer.IntraAnnualSlicesDefaultto add flexibility for users)files containing new code {new lines are commented with #$$}:
save_validation_config()netcdf.Datasetqa4sm_reader_versionto metadata (doesn't the environment.yaml file need to be adapted too when publishing the branch?)validator.validation.netcdf_transcription.Pytesmo2Qa4smResultsTranscriberinstancecreate_pytesmo_validation():run_validation():check_results(),validator.validation.netcdf_transcription.Pytesmo2Qa4smResultsTranscriberinstance. Accordingly, all occurrences of this method throughout this file were adaptedcheck_results()and added a code block prior to it, instantiating a transcriber instance. Note: currently the tests are only adapted to run for the bulk scenario. I did not add the #$$ comment to all 20 occurrences, though....test_errors(): l 1560-1566Outlook: