Closed JohnHalleyGotway closed 2 months ago
Note that all of the differences flagged in this testing workflow run are unrelated to these changes. They are remaining diffs from @hsoh-u recent PR that modified the UGRID output. We need to update develop-ref
to move past them, but @hsoh-u is making changes to reduce the overall number of new/modified files.
egrep -i "warning|error|file1:" comp_dir.log | egrep -B 1 ERROR | grep file1
file1: /data/output/met_test_truth/grid_stat_ugrid/grid_stat_UGRID_MPAS_DIAG_000000L_20120409_120000V.stat
file1: /data/output/met_test_truth/grid_stat_ugrid/grid_stat_UGRID_MPAS_DIAG_000000L_20120409_120000V_pairs.nc
file1: /data/output/met_test_truth/grid_stat_ugrid/grid_stat_UGRID_MPAS_OUT_TO_GRID_000000L_20120409_120000V.stat
file1: /data/output/met_test_truth/grid_stat_ugrid/grid_stat_UGRID_MPAS_OUT_TO_GRID_000000L_20120409_120000V_pairs.nc
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_DIAG_TEMP_000000L_20120409_120000V.stat
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_DIAG_TEMP_000000L_20120409_120000V_cnt.txt
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_DIAG_TEMP_000000L_20120409_120000V_ctc.txt
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_DIAG_TEMP_000000L_20120409_120000V_cts.txt
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_DIAG_TEMP_000000L_20120409_120000V_eclv.txt
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_DIAG_TEMP_000000L_20120409_120000V_fho.txt
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_DIAG_TEMP_000000L_20120409_120000V_mpr.txt
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_OUT_TEMP_000000L_20120409_120000V.stat
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_OUT_TEMP_000000L_20120409_120000V_cnt.txt
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_OUT_TEMP_000000L_20120409_120000V_ctc.txt
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_OUT_TEMP_000000L_20120409_120000V_cts.txt
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_OUT_TEMP_000000L_20120409_120000V_eclv.txt
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_OUT_TEMP_000000L_20120409_120000V_fho.txt
file1: /data/output/met_test_truth/point_stat_ugrid/point_stat_UGRID_MPAS_OUT_TEMP_000000L_20120409_120000V_mpr.txt
This PR includes the following changes:
parse_file_list_type(const StringArray&)
utility function to loop through the input files and determine the file type of the first non-missing file.log_missing_file(...)
utility function to write a warning message about missing files or just DEBUG(3) if the missing file name begins with theMISSING
keyword from METplus wrappers.Note the following two tweaks to the logic:
Calling
parse_file_list_type(...)
instead enables the first file in the list to be missing. So that's an improvement.open()
on the first input file, and that may have applied a little extra validation logic. But that isn't REALLY needed here because we open the files later when we process them. Not opening them here is every so slightly more efficient but it's possible that if a runtime error was going to occur, it may have occurred sooner rather than later in the logic.Expected Differences
[x] Do these changes introduce new tools, command line arguments, or configuration file options? [No] If yes, please describe:
[x] Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No] If yes, please describe:
Pull Request Testing
Observe the following log messages... 2 DEBUG and 1 WARNING:
Observe the following log messages...
I'll note that providing a
-grid_obs
bad path as the first one in the list (-grid_obs /a/bath/path
) does result in a runtime error. So this PR doesn't solve all the issues with bad data inputs... just the ones described in the issue.[x] Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions: The code for this PR is compiled and available for testing as the
met_test
user inseneca:/d1/projects/MET/MET_pull_requests/met-12.0.0/beta5/MET-feature_2870_remove_MISSING_warning
.[x] Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [No] I couldn't decide whether or not documenting the subtle difference between a warning and debug log message for missing files that start with the
MISSING
keyword is really warranted. So I did NOT add a note about to the user's guide. Do you think I should?[x] Do these changes include sufficient testing updates? [Yes] I updated the existing
unit_gen_ens_prod.xml
file. It includes 3 references to agep3
file that does NOT actually exist. I increased the verbosity level to 3 for all these runs and updated those 3 references to:MISSING
MISSING/&DATA_DIR_MODEL;/grib1/arw-tom-gep3/arw-tom-gep3_2012040912_F024.grib
&DATA_DIR_MODEL;/grib1/arw-tom-gep3/arw-tom-gep3_2012040912_F024.grib
The first 2 should produce a DEBUG(3) log message and the 3rd should still produce a warning message.
[x] Will this PR result in changes to the MET test suite? [No] If yes, describe the new output and/or changes to the existing output:
[x] Will this PR result in changes to existing METplus Use Cases? [No] If yes, create a new Update Truth METplus issue to describe them. I will change the log output from METplus ensemble use cases, but not the actual output.
[x] Do these changes introduce new SonarQube findings? [No] If yes, please describe: This PR passed the SonarQube quality gate check.
[x] Please complete this pull request review by [Fri 5/3/24].
Pull Request Checklist
See the METplus Workflow for details.