Update setting of environment variables for MET config files to add support for all to METPLUS_ vars

[georgemccabe]

We are moving towards a new format of setting environment variables that are read by the wrapped MET config files.

• All environment variables set that are used in the wrapped files should start with METPLUS_, i.e. METPLUS_MODEL. These variables should either contain an empty string (if not used so the default value will be used) or set to a string that contains the MET config variable name, an equals sign, the value, and a semi-colon at the end, i.e.

METPLUS_MODEL = model = "WRF"; or METPLUS_MODEL = ""

• We should continue to support the old naming of environment variables for a few releases so that users' wrapped MET config files that use the old naming do not need to be updated immediately upon switching to v4.0.0.

• We should keep track of the list of METPLUS_ variables that are used for each wrapper and output a warning log message if any of them are not currently being utilized in the user's wrapped MET config file so that they are aware that they can update their file to add additional capability.

• We should add a "MET Configuration" section to the Python Wrappers page in the User's Guide to describe how each environment variable corresponds to the METplus configuration variables. Currently there are a couple examples that have this information. Here is one: https://dtcenter.github.io/METplus/latest/Users_Guide/wrappers.html#tc-rmw-met-conf

• Once the information about the env vars set for the MET config files are in the Python Wrappers section, we should remove that content from each use case documentation file and instead provide a link to the appropriate section in the Python Wrappers section so that this information does not need to be updated in multiple places.

Describe the Enhancement

Engineering: Modifications to each wrapper should be made to

• support the new env vars (METPLUS_)
• Ensure old env vars are still set properly
• Add warnings if any METPLUS_ variable is not utilized in the MET config file used

Documentation:

• Add information on how each METPLUS_ variable is set for each wrapper in the Python Wrappers section
• Update use case docs to link to the appropriate section in Python Wrappers instead of repeating info about MET config variables

Time Estimate

~5 days

Sub-Issues

Consider breaking the enhancement down into sub-issues.

• [ ] Engineering Changes
• [ ] Documentation Updates

Relevant Deadlines

4.0.0

Funding Source

2790541 (no longer valid) Now split between 2702691, 2788881, 2799991

Define the Metadata

Assignee

• [X] Select engineer(s) or no engineer required
• [X] Select scientist(s) or no scientist required

Labels

• [X] Select component(s)
• [X] Select priority
• [X] Select requestor(s)

Projects and Milestone

• [X] Review projects and select relevant Repository and Organization ones or add "alert:NEED PROJECT ASSIGNMENT" label
• [X] Select milestone to next major version milestone or "Future Versions"

Define Related Issue(s)

Consider the impact to the other METplus components.

• [X] METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy

Enhancement Checklist

See the METplus Workflow for details.

• [x] Complete the issue definition above, including the Time Estimate and Funding Source.
• [x] Fork this repository or create a branch of develop. Branch name: feature_<Issue Number>_<Description>
• [x] Complete the development and test your changes.
• [x] Add/update log messages for easier debugging.
• [x] Add/update unit tests.
• [x] Add/update documentation.
• [x] Push local changes to GitHub.
• [x] Submit a pull request to merge into develop. Pull request: feature <Issue Number> <Description>
• [x] Define the pull request metadata, as permissions allow. Select: Reviewer(s), Project(s), Milestone, and Linked issues
• [x] Iterate until the reviewer(s) accept and merge your changes.
• [x] Delete your fork or branch.
• [x] Close this issue.

[georgemccabe]

Dan and I are working on branch feature_768_metconfig I have finished implementing support for all new METPLUS environment variables in the following wrappers:

ASCII2NC EnsembleStat GridStat MODE MTD PointStat

Changes made to these wrappers are easy to replicate in the other wrappers. Some variables have already been supported in the other wrappers with changes to be consistent across wrappers.

Still need to finish:

GridDiag PB2NC SeriesAnalysis STATAnalysis TCGen TCPairs TCRMW TCStat

[georgemccabe]

Finished: GridDiag PB2NC SeriesAnalysis STATAnalysis TCGen TCRMW

To go: TCPairs TCStat

Still need to test changes against previous version to ensure output is the same and update docs

[georgemccabe]

All wrappers have now implemented the new method for setting environment variables.

[georgemccabe]

@DanielAdriaansen I added a check for unused environment variables in the wrapped MET config files. Any item that is in the wrappers WRAPPER_ENV_VAR_KEYS list that is not found in the wrapped MET config file will produce a warning:

(command_builder.py:146) WARNING: Environment variable ${METPLUS_FCST_FIELD} is not utilized in MET config file: /d1/personal/mccabe/METplus/parm/met_config/PointStatConfig_wrapped
(command_builder.py:146) WARNING: Environment variable ${METPLUS_OBS_FIELD} is not utilized in MET config file: /d1/personal/mccabe/METplus/parm/met_config/PointStatConfig_wrapped
(command_builder.py:146) WARNING: Environment variable ${METPLUS_MESSAGE_TYPE} is not utilized in MET config file: /d1/personal/mccabe/METplus/parm/met_config/PointStatConfig_wrapped
(command_builder.py:146) WARNING: Environment variable ${METPLUS_CLIMO_MEAN_FILE} is not utilized in MET config file: /d1/personal/mccabe/METplus/parm/met_config/PointStatConfig_wrapped
(command_builder.py:146) WARNING: Environment variable ${METPLUS_CLIMO_STDEV_FILE} is not utilized in MET config file: /d1/personal/mccabe/METplus/parm/met_config/PointStatConfig_wrapped
(command_builder.py:146) WARNING: Environment variable ${METPLUS_OBS_WINDOW_DICT} is not utilized in MET config file: /d1/personal/mccabe/METplus/parm/met_config/PointStatConfig_wrapped
(command_builder.py:146) WARNING: Environment variable ${METPLUS_MASK_GRID} is not utilized in MET config file: /d1/personal/mccabe/METplus/parm/met_config/PointStatConfig_wrapped
(command_builder.py:146) WARNING: Environment variable ${METPLUS_MASK_POLY} is not utilized in MET config file: /d1/personal/mccabe/METplus/parm/met_config/PointStatConfig_wrapped
(command_builder.py:146) WARNING: Environment variable ${METPLUS_MASK_SID} is not utilized in MET config file: /d1/personal/mccabe/METplus/parm/met_config/PointStatConfig_wrapped
(command_builder.py:146) WARNING: Environment variable ${METPLUS_OUTPUT_PREFIX} is not utilized in MET config file: /d1/personal/mccabe/METplus/parm/met_config/PointStatConfig_wrapped

This should make it easier for you to see what has not yet been updated in the files in parm/met_config. Keep in mind that if I have forgotten to add a variable name to the wrapper's list, then it will not show up here.

[georgemccabe]

The diff output from running develop and this branch can be found on kiowa here:

/d1/personal/mccabe/diff-output/diff.log

The differences in the CyclonePlotter text file are due to the wrapper writing the output in a different order each run, which will be resolved with this PR: #797

The differences in the TCStat output have been resolved in a config change on the feature branch feature_768_met_config

[DanielAdriaansen]

All met_tool_wrapper GridStat use cases were tested with the new wrapped GridStat config file and no differences between the output of each use case using the wrapped GridStat config file from this branch and the wrapped GridStat config file from the develop branch were noted. Documentation for the met_tool_wrapper GridStat use cases has been updated. Next up will be testing model_applications GridStat use cases:

GridStat_fcstCESM_obsGFS_ConusTemp.py
GridStat_fcstHRRR_obsPracPerfect_SurrogateSevereProb.py
GridStat_fcstHRRR_obsPracPerfect_SurrogateSevere.py
GridStat_MODE_fcstIMS_obsNCEP_sea_ice.py
GridStat_fcstGFS_obsGFS_climoNCEP_MultiField.py
GridStat_fcstGFS_obsGFS_Sfc_MultiField.py
GridStat_fcstGFS_obsCCPA_GRIB.py
GridStat_fcstHREFmean_obsStgIV_Gempak.py
GridStat_fcstHREFmean_obsStgIV_NetCDF.py
GridStat_fcstHRRR-TLE_obsStgIV_GRIB.py
GridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.py
GridStat_fcstGloTEC_obsGloTEC_vx7.py

[DanielAdriaansen]

All of the GridStat model_applications use cases have been updated in two places:

1. Each METplus conf file was switched to use the GridStatConfig_wrapped MET config file
2. Each METplus docs Python/RST file was updated so the literalinclude is including the GridStatConfig_wrappped MET config file
3. Each METplus conf file for the use case was switched from using CONFIG_DIR to PARM_BASE/met_config (for GRID_STAT_MET_CONFIG only)

No testing was done on these changes locally. I will be moving to the next wrapper now.

A checklist of sorts based on my experience with GridStat:

• [ ] Make sure the _wrapped config file for the MET tool has been updated to include all of the METPLUS_* env vars
• [ ] Change the literalinclude in the use case docs file to be the wrapped MET config file
• [ ] Change the paragraph under "MET configuration" to match for all use cases (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/generated/met_tool_wrapper/GridStat/GridStat.html#met-configuration)
• [ ] Make any other changes to the use case documentation file and cleanup that might be needed since we are in there
• [ ] Change the METplus config file for the use case to use the wrapped MET config file, making sure to switch from CONFIG_DIR to PARM_BASE/met_config in the path
• [ ] Clean up the documentation in the wrappers.rst file for the wrapper being updated to match the style/formatting of GridStat (seen here: https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#gridstat)
• [ ] Remove families from list of conf items for this wrapper
• [ ] Update MET Configuration section (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#grid-stat-met-conf)

[DanielAdriaansen]

Checklist for each wrapper:

• [x] GridStat
• [x] EnsembleStat
• [x] ASCII2NC
• [x] GridDiag
• [x] MODE
• [ ] MTD
• [x] PB2NC
• [x] PointStat
• [x] SeriesAnalysis
• [x] StatAnalysis
• [x] TcGen
• [x] TcPairs
• [x] TcRMW
• [x] TcStat

[DanielAdriaansen]

Update- it looks like the GridStat S2S model_applications use case needs some attention. I think some output_flag or nc_pairs_flag changes are needed to get the use case to work properly. I tried testing with GRID_STAT_MET_CONFIG_OVERRIDES but I can't get that to work presently.

[DanielAdriaansen]

With the help of George all GridStat use cases are passing now. I am moving to EnsembleStat.

• [x] Make sure the _wrapped config file for the MET tool has been updated to include all of the METPLUS_* env vars
• [x] Change the literalinclude in both met_tool_wrapper and model_applications use case docs files to be the wrapped MET config file
• [x] Change the paragraph under "MET configuration" to match for all use cases (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/generated/met_tool_wrapper/GridStat/GridStat.html#met-configuration)
• [x] Make any other changes to the use case documentation files and cleanup that might be needed since we are in there
• [x] Change the METplus config file for the met_tool_wrapper and model_applications use cases to use the wrapped MET config file, making sure to switch from CONFIG_DIR to PARM_BASE/met_config in the path
• [x] Clean up the documentation in the wrappers.rst file for the wrapper being updated to match the style/formatting of GridStat (seen here: https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#gridstat)
• [x] Remove families from list of conf items for this wrapper
• [x] Update MET Configuration section (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#grid-stat-met-conf)
• [x] Add all of the new METPLUS_* items to the wrappers RST docs file under the appropriate wrappers section following the GridStat wrapper section tabular formatting.

[DanielAdriaansen]

EnsembleStat use cases have been updated and docs changes are complete. Here are a list of use cases changed:

met_tool_wrapper/EnsembleStat/EnsembleStat.conf
met_tool_wrapper/EnsembleStat/EnsembleStat_python_embedding.conf
model_applications/air_quality_and_comp/EnsembleStat_fcstICAP_obsMODIS_aod.conf
model_applications/convection_allowing_models/EnsembleStat_fcstHRRRE_obsHRRRE_Sfc_MultiField.conf
model_applications/convection_allowing_models/EnsembleStat_fcstHRRR_fcstOnly_SurrogateSevere.conf
model_applications/precipitation/EnsembleStat_fcstHRRRE_FcstOnly_NetCDF.conf
model_applications/precipitation/EnsembleStat_fcstWOFS_obsWOFS.conf

The docs build except for the glossary entry for ENS_ENSEMBLE_STAT_DATATYPE (see #828).

[DanielAdriaansen]

Moving to ASCII2NC:

• [x] Make sure the _wrapped config file for the MET tool has been updated to include all of the METPLUS_* env vars
• [x] Change the literalinclude in the use case docs file to be the wrapped MET config file
• [x] Change the paragraph under "MET configuration" to match for all use cases (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/generated/met_tool_wrapper/GridStat/GridStat.html#met-configuration)
• [x] Make any other changes to the use case documentation file and cleanup that might be needed since we are in there
• [x] Change the METplus config file for the use case to use the wrapped MET config file, making sure to switch from CONFIG_DIR to PARM_BASE/met_config in the path
• [x] Clean up the documentation in the wrappers.rst file for the wrapper being updated to match the style/formatting of GridStat (seen here: https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#gridstat)
• [x] Remove families from list of conf items for this wrapper
• [x] Update MET Configuration section (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#grid-stat-met-conf)
• [x] Verify docs build locally

[DanielAdriaansen]

ASCII2NC was a breeze, and only had met_toolwrapper use cases and one METPLUS* item to support. Here are the modified use cases:

met_tool_wrapper/ASCII2NC/ASCII2NC.conf

The docs build except for previously mentioned errors.

[DanielAdriaansen]

Moving to GridDiag:

• [x] Make sure the _wrapped config file for the MET tool has been updated to include all of the METPLUS_* env vars
• [x] Change the literalinclude in the use case docs file to be the wrapped MET config file
• [x] Change the paragraph under "MET configuration" to match for all use cases (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/generated/met_tool_wrapper/GridStat/GridStat.html#met-configuration)
• [x] Make any other changes to the use case documentation file and cleanup that might be needed since we are in there
• [x] Change the METplus config file for the use case to use the wrapped MET config file, making sure to switch from CONFIG_DIR to PARM_BASE/met_config in the path
• [x] Clean up the documentation in the wrappers.rst file for the wrapper being updated to match the style/formatting of GridStat (seen here: https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#gridstat)
• [x] Remove families from list of conf items for this wrapper
• [x] Update MET Configuration section (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#grid-stat-met-conf)
• [x] Verify docs build locally

[georgemccabe]

I have started some of the MODE examples, so I would skip that tool for now.

• [X] Make sure the _wrapped config file for the MET tool has been updated to include all of the METPLUS_* env vars
• [x] Change the literalinclude in the use case docs file to be the wrapped MET config file
• [x] Change the paragraph under "MET configuration" to match for all use cases (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/generated/met_tool_wrapper/GridStat/GridStat.html#met-configuration)
• [x] Make any other changes to the use case documentation file and cleanup that might be needed since we are in there
• [X] Change the METplus config file for the use case to use the wrapped MET config file, making sure to switch from CONFIG_DIR to PARM_BASE/met_config in the path
• [x] Clean up the documentation in the wrappers.rst file for the wrapper being updated to match the style/formatting of GridStat (seen here: https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#gridstat)
• [x] Remove families from list of conf items for this wrapper
• [x] Update MET Configuration section (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#grid-stat-met-conf)
• [x] Verify docs build locally

[DanielAdriaansen]

GridDiag similarly straightforward, mainly due to lack of model_applications use cases. Here are the modified use cases:

met_tool_wrapper/GridDiag/GridDiag.conf

Docs build successfully.

[DanielAdriaansen]

Moving to PB2NC:

• [x] Make sure the _wrapped config file for the MET tool has been updated to include all of the METPLUS_* env vars
• [x] Change the literalinclude in the use case docs file to be the wrapped MET config file
• [x] Change the paragraph under "MET configuration" to match for all use cases (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/generated/met_tool_wrapper/GridStat/GridStat.html#met-configuration)
• [x] Make any other changes to the use case documentation file and cleanup that might be needed since we are in there
• [x] Change the METplus config file for the use case to use the wrapped MET config file, making sure to switch from CONFIG_DIR to PARM_BASE/met_config in the path
• [x] Clean up the documentation in the wrappers.rst file for the wrapper being updated to match the style/formatting of GridStat (seen here: https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#gridstat)
• [x] Remove families from list of conf items for this wrapper
• [x] Update MET Configuration section (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#grid-stat-met-conf)
• [x] Verify docs build locally

[DanielAdriaansen]

PB2NC was also straightforward. Here are the modified use cases:

use_cases/met_tool_wrapper/PB2NC/PB2NC.conf

Docs build successfully.

[DanielAdriaansen]

Moving to PointStat:

• [x] Make sure the _wrapped config file for the MET tool has been updated to include all of the METPLUS_* env vars
• [x] Change the literalinclude in the use case docs file to be the wrapped MET config file
• [x] Change the paragraph under "MET configuration" to match for all use cases (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/generated/met_tool_wrapper/GridStat/GridStat.html#met-configuration)
• [x] Make any other changes to the use case documentation file and cleanup that might be needed since we are in there
• [x] Change the METplus config file for the use case to use the wrapped MET config file, making sure to switch from CONFIG_DIR to PARM_BASE/met_config in the path
• [x] Clean up the documentation in the wrappers.rst file for the wrapper being updated to match the style/formatting of GridStat (seen here: https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#gridstat)
• [x] Remove families from list of conf items for this wrapper
• [x] Update MET Configuration section (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#grid-stat-met-conf)
• [x] Verify docs build locally

[DanielAdriaansen]

PointStat is complete. Automation is running now. Here are the modified use cases:

met_tool_wrapper/PointStat/PointStat.conf
met_tool_wrapper/PointStat/PointStat_once_per_field.conf
model_applications/medium_range/PointStat_fcstGFS_obsGDAS_UpperAir_MultiField_PrepBufr.conf
model_applications/medium_range/PointStat_fcstGFS_obsNAM_Sfc_MultiField_PrepBufr.conf
model_applications/tc_and_extra_tc/UserScript_ASCII2NC_PointStat_fcstHAFS_obsFRD_NetCDFci_overrides.conf

Docs build.

[DanielAdriaansen]

Moving to StatAnalysis:

• [x] Make sure the _wrapped config file for the MET tool has been updated to include all of the METPLUS_* env vars
• [x] Change the literalinclude in the use case docs file to be the wrapped MET config file
• [x] Change the paragraph under "MET configuration" to match for all use cases (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/generated/met_tool_wrapper/GridStat/GridStat.html#met-configuration)
• [x] Make any other changes to the use case documentation file and cleanup that might be needed since we are in there
• [x] Change the METplus config file for the use case to use the wrapped MET config file, making sure to switch from CONFIG_DIR to PARM_BASE/met_config in the path
• [x] Clean up the documentation in the wrappers.rst file for the wrapper being updated to match the style/formatting of GridStat (seen here: https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#gridstat)
• [x] Remove families from list of conf items for this wrapper
• [x] Update MET Configuration section (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#grid-stat-met-conf)
• [x] Verify docs build locally

[DanielAdriaansen]

StatAnalysis is complete. All use cases appear to be passing automation. Here are the modified use cases:

met_tool_wrapper/StatAnalysis/StatAnalysis.conf
met_tool_wrapper/StatAnalysis/StatAnalysis_python_embedding.conf
model_applications/data_assimilation/StatAnalysis_fcstHAFS_obsPrepBufr_JEDI_IODA_interface.conf

Docs build- some issues with items not in conf glossary. Will resolve later.

[DanielAdriaansen]

Moving to SeriesAnalysis:

• [x] Make sure the _wrapped config file for the MET tool has been updated to include all of the METPLUS_* env vars
• [x] Change the literalinclude in the use case docs file to be the wrapped MET config file
• [x] Change the paragraph under "MET configuration" to match for all use cases (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/generated/met_tool_wrapper/GridStat/GridStat.html#met-configuration)
• [x] Make any other changes to the use case documentation file and cleanup that might be needed since we are in there
• [x] Change the METplus config file for the use case to use the wrapped MET config file, making sure to switch from CONFIG_DIR to PARM_BASE/met_config in the path
• [x] Clean up the documentation in the wrappers.rst file for the wrapper being updated to match the style/formatting of GridStat (seen here: https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#gridstat)
• [x] Remove families from list of conf items for this wrapper
• [x] Update MET Configuration section (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#grid-stat-met-conf)
• [x] Verify docs build locally

[DanielAdriaansen]

SeriesAnalysis is complete. Automation has not run yet. Here are the modified use cases:

met_tool_wrapper/SeriesAnalysis/SeriesAnalysis.conf
met_tool_wrapper/SeriesAnalysis/SeriesAnalysis_python_embedding.conf
model_applications/s2s/GridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast.conf

Note that it appears all SeriesAnalysis use cases were already using the _wrapped MET config file, and that file already contained all the new METPLUS_* items supported by series_analysis_wrapper so theoretically these changes are only for documentation and all use cases should still run.

Docs build- some issues with items not in conf glossary. Will resolve later.

[DanielAdriaansen]

Moving to TcGen:

• [x] Make sure the _wrapped config file for the MET tool has been updated to include all of the METPLUS_* env vars
• [x] Change the literalinclude in the use case docs file to be the wrapped MET config file
• [x] Change the paragraph under "MET configuration" to match for all use cases (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/generated/met_tool_wrapper/GridStat/GridStat.html#met-configuration)
• [x] Make any other changes to the use case documentation file and cleanup that might be needed since we are in there
• [x] Change the METplus config file for the use case to use the wrapped MET config file, making sure to switch from CONFIG_DIR to PARM_BASE/met_config in the path
• [x] Clean up the documentation in the wrappers.rst file for the wrapper being updated to match the style/formatting of GridStat (seen here: https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#gridstat)
• [x] Remove families from list of conf items for this wrapper
• [x] Update MET Configuration section (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#grid-stat-met-conf)
• [x] Verify docs build locally

[DanielAdriaansen]

TCGen is complete. Automation has not run yet. Here are the modified use cases:

met_tool_wrapper/TCGen/TCGen.conf

Note that it appears all TCGen use cases were already using the _wrapped MET config file, and that file already contained all the new METPLUS_* items supported by tc_gen_wrapper so theoretically these changes are only for documentation and all use cases should still run.

Docs build- some issues with items not in conf glossary. Will resolve later.

NOTE- these changes align with METv9.1 When #801 is complete this section will need to be updated appropriately.

[DanielAdriaansen]

Moving to TcPairs:

• [x] Make sure the _wrapped config file for the MET tool has been updated to include all of the METPLUS_* env vars
• [x] Change the literalinclude in the use case docs file to be the wrapped MET config file
• [x] Change the paragraph under "MET configuration" to match for all use cases (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/generated/met_tool_wrapper/GridStat/GridStat.html#met-configuration)
• [x] Make any other changes to the use case documentation file and cleanup that might be needed since we are in there
• [x] Change the METplus config file for the use case to use the wrapped MET config file, making sure to switch from CONFIG_DIR to PARM_BASE/met_config in the path
• [x] Clean up the documentation in the wrappers.rst file for the wrapper being updated to match the style/formatting of GridStat (seen here: https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#gridstat)
• [x] Remove families from list of conf items for this wrapper
• [x] Update MET Configuration section (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#grid-stat-met-conf)
• [x] Verify docs build locally

[DanielAdriaansen]

TCPairs is complete. Automation has not run yet. Here are the modified use cases:

met_tool_wrapper/TCPairs/TCPairs_extra_tropical.py
met_tool_wrapper/TCPairs/TCPairs_tropical.py
model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.py
model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_RPlotting.py

Docs build- some issues with items not in conf glossary. Will resolve later.

The use cases using TCPairs should be monitored because the _wrapped MET config file was changed for these use cases.

[DanielAdriaansen]

Moving to TcRMW:

• [ ] Make sure the _wrapped config file for the MET tool has been updated to include all of the METPLUS_* env vars
• [ ] Change the literalinclude in the use case docs file to be the wrapped MET config file
• [ ] Change the paragraph under "MET configuration" to match for all use cases (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/generated/met_tool_wrapper/GridStat/GridStat.html#met-configuration)
• [ ] Make any other changes to the use case documentation file and cleanup that might be needed since we are in there
• [ ] Change the METplus config file for the use case to use the wrapped MET config file, making sure to switch from CONFIG_DIR to PARM_BASE/met_config in the path
• [ ] Clean up the documentation in the wrappers.rst file for the wrapper being updated to match the style/formatting of GridStat (seen here: https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#gridstat)
• [ ] Remove families from list of conf items for this wrapper
• [ ] Update MET Configuration section (e.g. https://ral.ucar.edu/staff/dadriaan/work/metconf/Users_Guide/wrappers.html#grid-stat-met-conf)
• [ ] Compare wrapped MET config file with any use-case specific MET config files and reconcile differences
• [ ] Verify docs build locally

[georgemccabe]

While doing this work, I learned that the values for fcst_raw_plot.color_table and the equivalent obs color table were incorrect in the parm/met_config/MODEConfig_wrapped file (differs from share/met/config version). This will result in differences in ps output. I have seen this a couple cases but there may be more. Once we get a full list of use cases that are affected, we can rerun them with the old value to confirm that no differences arise to ensure that the changes are expected.

Use cases that should have no differences if the color scale is changed back to the wrong default (may not be a complete list):

• met_tool_wrapper/MODE/MODE.conf
• model_applications/convection_allowing_models/MODE_fcstHRRR_obsMRMS_Hail_GRIB2.conf
• model_applications/convection_allowing_models/MODE_fcstFV3_obsGOES_BrightnessTemp.conf

The following run includes overriding the color scale to the old value for these 3 cases. If no differences occur in result, then we can safely assume that the output of these 3 use cases will differ and that is expected: https://github.com/dtcenter/METplus/actions/runs/625210865 We will need to update the develop-ref branch from develop after these changes are merged in.

[georgemccabe]

Update: The following are output in the difference output that are expected changes:

Caused by incorrect default color table in default wrapped MET config:

met_tool_wrapper

Text diff A:/data/truth/MODE/mode/2005080712/mode_WRF_RH_vs_WRF_RH_P500_120000L_20050807_120000V_000000A.ps B:/data/output/MODE/mode/2005080712/mode_WRF_RH_vs_WRF_RH_P500_120000L_20050807_120000V_000000A.ps

convection_allowing_models:0-5

Text diff A:/data/truth/MODE_fcstHRRR_obsMRMS_Hail_GRIB2/hailtest/mode_270000L_20190529_030000V_010000A.ps B:/data/output/MODE_fcstHRRR_obsMRMS_Hail_GRIB2/hailtest/mode_270000L_20190529_030000V_010000A.ps Text diff A:/data/truth/MODE_fcstHRRR_obsMRMS_Hail_GRIB2/hailtest/mode_260000L_20190529_020000V_010000A.ps B:/data/output/MODE_fcstHRRR_obsMRMS_Hail_GRIB2/hailtest/mode_260000L_20190529_020000V_010000A.ps

convection_allowing_models:6+

Text diff A:/data/truth/MODE_fcstFV3_obsGOES_BrightnessTemp/convection_allowing_models/brightness_temperature/mode_FV3_core_lsm1_020000L_20190521_020000V_NAA.ps B:/data/output/MODE_fcstFV3_obsGOES_BrightnessTemp/convection_allowing_models/brightness_temperature/mode_FV3_core_lsm1_020000L_20190521_020000V_NAA.ps

New output from new use cases (PointStat_python_embedding and MODE_python_embedding) that are not found in truth data set

met_tool_wrapper

file not found (new output) A: B:/data/output/PointStat_python_embedding/metplus_final.conf file not found (new output) A: B:/data/output/PointStat_python_embedding/point_stat_py_embed/point_stat_000000L_20200906_060000V.stat file not found (new output) A: B:/data/output/MODE_python_emedding/metplus_final.conf file not found (new output) A: B:/data/output/MODE_python_emedding/met_tool_wrapper/MODE_python_embedding/2005080712/mode_FCST_vs_OBS_120000L_20050807_120000V_120000A_obj.nc file not found (new output) A: B:/data/output/MODE_python_emedding/met_tool_wrapper/MODE_python_embedding/2005080712/mode_FCST_vs_OBS_120000L_20050807_120000V_120000A_cts.txt file not found (new output) A: B:/data/output/MODE_python_emedding/met_tool_wrapper/MODE_python_embedding/2005080712/mode_FCST_vs_OBS_120000L_20050807_120000V_120000A_obj.txt file not found (new output) A: B:/data/output/MODE_python_emedding/met_tool_wrapper/MODE_python_embedding/2005080712/mode_FCST_vs_OBS_120000L_20050807_120000V_120000A.ps

Extra masking fields were generated mistakenly

We need @lindsayrblank to confirm that the extra fields are not needed, but I believe this to be the case. The settings for this use case had an empty string set for the mask.grid, i.e.:

mask = {
   grid = [ "" ];
  poly = [ ${VERIF_MASK} ];
}

This should actually be set to:

grid = [];

because we want to only use the poly values to determine the grid. All of the downstream output from MODE is identical to the truth data, which implies that the extra mask output fields are not being used in the use case.

COMPARING GridStat_MODE_fcstIMS_obsNCEP_sea_ice/model_applications/cryosphere/sea_ice/GridStat/20190201/grid_stat/grid_stat_IMS_ICEC_vs_NCEP_ICEC_Z0_000000L_20190201_220000V.stat file_A: /data/truth/GridStat_MODE_fcstIMS_obsNCEP_sea_ice/model_applications/cryosphere/sea_ice/GridStat/20190201/grid_stat/grid_stat_IMS_ICEC_vs_NCEP_ICEC_Z0_000000L_20190201_220000V.stat file_B: /data/output/GridStat_MODE_fcstIMS_obsNCEP_sea_ice/model_applications/cryosphere/sea_ice/GridStat/20190201/grid_stat/grid_stat_IMS_ICEC_vs_NCEP_ICEC_Z0_000000L_20190201_220000V.stat

Comparing text files Comparing stat file ERROR: Different number of lines in /data/output/GridStat_MODE_fcstIMS_obsNCEP_sea_ice/model_applications/cryosphere/sea_ice/GridStat/20190201/grid_stat/grid_stat_IMS_ICEC_vs_NCEP_ICEC_Z0_000000L_20190201_220000V.stat File_A: 18 File_B: 9 ERROR: File differs: /data/output/GridStat_MODE_fcstIMS_obsNCEP_sea_ice/model_applications/cryosphere/sea_ice/GridStat/20190201/grid_stat/grid_stat_IMS_ICEC_vs_NCEP_ICEC_Z0_000000L_20190201_220000V.stat

COMPARING GridStat_MODE_fcstIMS_obsNCEP_sea_ice/model_applications/cryosphere/sea_ice/GridStat/20190201/grid_stat/grid_stat_IMS_ICEC_vs_NCEP_ICEC_Z0_000000L_20190201_220000V_pairs.nc file_A: /data/truth/GridStat_MODE_fcstIMS_obsNCEP_sea_ice/model_applications/cryosphere/sea_ice/GridStat/20190201/grid_stat/grid_stat_IMS_ICEC_vs_NCEP_ICEC_Z0_000000L_20190201_220000V_pairs.nc file_B: /data/output/GridStat_MODE_fcstIMS_obsNCEP_sea_ice/model_applications/cryosphere/sea_ice/GridStat/20190201/grid_stat/grid_stat_IMS_ICEC_vs_NCEP_ICEC_Z0_000000L_20190201_220000V_pairs.nc

Comparing NetCDF ERROR: Field list differs between files File_A: ['DIFF_ICEC_Z0_ICECZ0', 'DIFF_ICEC_Z0_ICEC_Z0_data_mask', 'FCST_ICECZ0', 'FCST_ICEC_Z0eq1.0_NBRHD_25', 'FCST_ICEC_Z0eq1.0_NBRHD_49', 'FCST_ICEC_Z0eq1.0_NBRHD_81', 'FCST_ICEC_Z0eq1.0_NBRHD_9', 'FCST_ICEC_Z0_data_mask', 'FCST_ICEC_Z0_data_mask_eq1.0_NBRHD_25', 'FCST_ICEC_Z0_data_mask_eq1.0_NBRHD_49', 'FCST_ICEC_Z0_data_mask_eq1.0_NBRHD_81', 'FCST_ICEC_Z0_data_mask_eq1.0_NBRHD_9', 'OBS_ICECZ0', 'OBS_ICEC_Z0ge0.40_NBRHD_25', 'OBS_ICEC_Z0ge0.40_NBRHD_49', 'OBS_ICEC_Z0ge0.40_NBRHD_81', 'OBS_ICEC_Z0ge0.40_NBRHD_9', 'OBS_ICEC_Z0_data_mask', 'OBS_ICEC_Z0_data_mask_ge0.40_NBRHD_25', 'OBS_ICEC_Z0_data_mask_ge0.40_NBRHD_49', 'OBS_ICEC_Z0_data_mask_ge0.40_NBRHD_81', 'OBS_ICEC_Z0_data_mask_ge0.40_NBRHD_9', 'lat', 'lon'] File_B:['DIFF_ICEC_Z0_ICEC_Z0_data_mask', 'FCST_ICEC_Z0_data_mask', 'FCST_ICEC_Z0_data_mask_eq1.0_NBRHD_25', 'FCST_ICEC_Z0_data_mask_eq1.0_NBRHD_49', 'FCST_ICEC_Z0_data_mask_eq1.0_NBRHD_81', 'FCST_ICEC_Z0_data_mask_eq1.0_NBRHD_9', 'OBS_ICEC_Z0_data_mask', 'OBS_ICEC_Z0_data_mask_ge0.40_NBRHD_25', 'OBS_ICEC_Z0_data_mask_ge0.40_NBRHD_49', 'OBS_ICEC_Z0_data_mask_ge0.40_NBRHD_81', 'OBS_ICEC_Z0_data_mask_ge0.40_NBRHD_9', 'lat', 'lon'] Using File_A fields. ERROR: Field DIFF_ICEC_Z0_ICECZ0 not found

No output lines in truth data

precipitation

Text diff A:/data/truth/GridStat_fcstGFS_obsCCPA_GRIB/model_applications/precipitation/GridStat_fcstGFS_obsCCPA_GRIB/met_out/GFS/precip/201706130000/grid_stat/grid_stat_GFS_APCP_vs_ANLYS_APCP_A24_240000L_20170613_000000V.stat B:/data/output/GridStat_fcstGFS_obsCCPA_GRIB/model_applications/precipitation/GridStat_fcstGFS_obsCCPA_GRIB/met_out/GFS/precip/201706130000/grid_stat/grid_stat_GFS_APCP_vs_ANLYS_APCP_A24_240000L_20170613_000000V.stat

Threshold values were removed from this use case, which caused there to be no output lines. 5 threshold values were added back and the use case now produces output.

[georgemccabe]

Full logs of the differences tests can be viewed here (for ~90 days): https://github.com/dtcenter/METplus/actions/runs/633804009

[DanielAdriaansen]

We should also update the docs about adding a new use case to note that a MET conf file is no longer needed- the _wrapped file should be used.

https://dtcenter.github.io/METplus/latest/Contributors_Guide/add_use_case.html