Closed mjs2369 closed 9 months ago
note to reviewers, the documentation build for this pull request is here: https://dart-documentation--553.org.readthedocs.build/en/553/
I think filter_kind needs removed from the description of namelist items on the assim_tools_mod documentation page. There is information about using Jon Potterjoy's Particle Filter there, though, that I'm not sure should be removed or where it should be moved to.
@hkershaw-brown I just retested this with all of the new code changes, and it is bitwise with quantile_methods for both the intel and gfortran compilers
This was tested for all lorenz_96_tracer_advection examples (tests A-D). Note that for tests B and C, I had to update the algorithm_info_mod code slightly as it was out-of-date and would not compile. You can see the types of changes made in this screenshot, fixing the bounds and using _DISTRIBUTION instead of _PRIOR:
This was also tested with the lorenz_96 model for sort_obs_inc = .true. AND .false (ensuring that the sorting code is actually running by using filter_kind=ENKF)
This was tested for all lorenz_96_tracer_advection examples (tests A-D). Note that for tests B and C, I had to update the algorithm_info_mod code slightly as it was out-of-date and would not compile.
@mjs2369 great thanks Marlee, can you go ahead and commit the changes to all_eakf_algorithm_info_mod and state_eakf_tracer_bnrhf_algorithm_info_mod to the quantile_methods branch. You should be able to push to that branch now.
@hkershaw-brown
Todo:
[x] rename qcf_table qceff table
[x] test lower case options in qcf-table
[x] test header version with column descriptions
[x] QCEFF paper 1 & 2
[x] test perfect_model_obs with non-default qty options. Perfect_model_obs is not logging the qceff table.
[x] perfect_model_obs remove use_algorithm_info_mod
[x] doc: remove/update namelist options that have been removed/deprecated/improved
[x] cam-fv input.nml remove new_qcf_input.nml (old version now!)
[x] defaults probit_transform
[x] defaults filter_kind (eakf), dist_type (normal_distribution)
[x] Jeff's edits to index.hml
todo:
check if (.not. module_initialized) call init_algorithm_info_mod(qcf_table_filename)
perfect_model_obs and filter call algorithm info mod, but only filter calls initalize with the qcf_table_filename. Is perfect_model_obs always using the default values for every qty?
@hkershaw-brown retested with all changes - bitwise identical with quantile_methods for all four lorenz_96_tracer_advection examples
I did one final review as well for both the code and docs and I think it is ready to merge!
Description:
This PR is a duplicate of #545 but with correctly revised git history.
The python script for converting from a YAML file to the table will be moved to a separate PR.
Previously the QCF code required an algorithm_info_mod specific to the model, which meant editing algorithm_info_mod.f90 to specify which distribution should be used for which quantity.
This code implements a QCF input table, which reads in the algorithm info choices (QCF options) at runtime and stores them in algorithm_info_mod module storage.
This replaces the former functionality of algorithm_info_mod if statements with the table information.
The observation, state, and inflation variables are read in from a single table. Each field keeps its own column, having 28 total in the table.
The full list of QCF input options and information of the structure of the table can be found in the documentation at DART/guide/qcf_table.rst
More info on the background of the issue can be read in the specification here: https://docs.google.com/document/d/1MnvEFfgj5SfFbnIahGHwjy1XJ5IWBvPS8NB1nrIjc8k/edit
Fixes issue
Fixes #503
Types of changes
Documentation changes needed?
While I have included new documentation on how to use the input table at DART/guide/qcf_table.rst , Jeff’s documentation at https://docs.dart.ucar.edu/en/quantile_methods/models/lorenz_96_tracer_advection/work/readme.html needs updated to reflect the difference in workflow for the tests listed, and in general the documentation for quantile methods should be highlighted and linked to on the main DART documentation page.
Tests
Compiled and ran filter with full debugging flags with Intel, CCE, gfortran Bitwise identical to quantile_methods, tested with Intel
Information on how to use the QCF input table with the quantile code is in the documentation at DART/guide/qcf_table.rst
build_everything now passes for all models:
Checklist for merging
Checklist for release
Testing Datasets