dtcenter / MET

Model Evaluation Tools
https://dtcenter.org/community-code/model-evaluation-tools-met
Apache License 2.0
74 stars 22 forks source link

Feature #2795 level_mismatch_warning #2873

Closed JohnHalleyGotway closed 2 months ago

JohnHalleyGotway commented 2 months ago

This PR affects the Point-Stat and Ensemble-Stat tools and refines the logic for when warning messages are printed based on NOAA/EMC feedback.

Previously, when processing the config files for these tools, the code checked to see if the requested forecast levels do not fully contain the observation levels... only for forecast pressure level types. The intention is to warn the user that vertical interpolation may not work as they expect.

The challenge is that a range of pressure levels (e.g. TMP/P500-850) may correspond to multiple individual levels... or it may correspond to a single record containing a LAYER of data. NOAA/EMC was seeing warnings when verifying a layer of data and would like those warnings to go away.

The change in this PR is to move the location of these checks. Rather than doing it when the config file is parsed, do it after the gridded data is read from the input forecast files. Change the logic to check this for ALL LEVEL TYPES (not just pressure levels) since vertical interpolation also works for height levels. But only check this if more than 1 forecast record was ACTUALLY read from the data. In NOAA/EMC's CAPE data case, only a single record will be found. So no warning will be printed.

Note as well, that Ensemble-Stat includes logic to only print this warning once for each verification task... rather than once for each ensemble member.

Expected Differences

Pull Request Testing

This makes the following 4 comparisons for temperature data in PointStatConfig:

fcst = {
   field = [ 
     { name = "TMP"; level = "L180-150"; },
     { name = "TMP"; level = "L180-150"; },
     { name = "TMP"; level = "P500-850"; },
     { name = "TMP"; level = "P500-850"; }
   ];
} 
obs = {
   field = [ 
     { name = "TMP"; level = "P180-150";  },
     { name = "TMP"; level = "P0-500";    },
     { name = "TMP"; level = "P500-850";  },
     { name = "TMP"; level = "P250-1000"; }
   ];
} 

It runs 2 versions of Point-Stat... one from the nightly build develop branch and once from my feature branch. Here's the difference in the log files:

< WARNING: 
< WARNING: PointStatVxOpt::process_config() -> The range of requested observation pressure levels is not contained within the range of requested forecast pressure levels.  No vertical interpolation will be performed for observations falling outside the range of forecast levels.  Instead, they will be matched to the single nearest forecast level.
< WARNING: 

> WARNING: 
> WARNING: process_fcst_climo_files() -> The forecast level range (TMP/P850-500) does not fully contain the observation level range (TMP/P1000-250). No vertical interpolation will be performed for observations falling outside the range of forecast levels. Instead, they will be matched to the single nearest forecast level.
> WARNING: 

Both are only for the 4th comparison. This admittedly isn't the most comprehensive test. But it does demonstrate the change to the warning message. And it points out that the new code does NOT trigger a warning for the 2nd task, where there's a mismatch between the forecast and observations levels. But since the forecast data is a single record with a layer of data, no warning message is triggered.

Pull Request Checklist

See the METplus Workflow for details.