Bugfix: Fix GridStat_SeriesAnalysis _fcstNMME_obsCPC _seasonal_forecast use cases with poorly configured climatology settings

JohnHalleyGotway commented 1 week ago

Describe the Problem

Merging pull request dtcenter/MET#2963 for the forecast climo config options of issue dtcenter/MET#2924 triggered this METplus Testing Workflow run. Two of the 56 use case groups had errors related to climatology data. Upon inspection, we found that the GridStat_SeriesAnalysis _fcstNMME_obsCPC _seasonal_forecast use case fails to supply the climo data as expected.

Expected Behavior

In Use Case Tests (s2s:0) the GridStat_SeriesAnalysis _fcstNMME_obsCPC _seasonal_forecast use case fails with the following error message:

egrep -i error series_analysis.full_stats.log | sort -u
ERROR  : Dictionary::lookup_string() -> lookup failed for name "level"

The issue is that the climo_mean file name is provided (on this line) but no field list is provided to specify what climo data should be read from that file. Fixing this may be as simple as adding the SERIES_ANALYSIS_CLIMO_MEAN_USE_FCST config option. However, you'd need to confirm that the forecast and climo data field lists really should use the same name and level settings.

Note that I did confirm that the existing use case IS NOT providing climo data as expected. I pulled the existing METplus truth data, ran a command to extract the Series-Analysis NetCDF output file, and used ncview to display it:

docker create --name output-s2s_0 dtcenter/metplus-data-dev:output-develop-s2s_0
docker run -it --rm --volumes-from output-s2s_0 -v `pwd`/window:/window  dtcenter/met:12.0.0-beta5 cp /data/truth/s2s/GridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast/model_applications/s2s/GridStat_SeriesAnalysis_fcstNMME_obsCPC_seasonal_forecast/SeriesAnalysis/series_analysis_NMME_CPC_stats_F2m_full_stats.nc /window/.
ncview window/series_analysis_NMME_CPC_stats_F2m_full_stats.nc

And the field of series_cnt_ANOM_CORR contains all bad data. That means that no climo mean data was actually passed to series_analysis.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

[ ] Select engineer(s) or no engineer required
[x] Select scientist(s) or no scientist required

Labels

[x] Review default alert labels
[x] Select component(s)
[x] Select priority
[x] Select requestor(s)

Milestone and Projects

[x] Select Milestone as the next bugfix version
[ ] Select Coordinated METplus-X.Y Support project for support of the current coordinated release
[x] Select METplus-Wrappers-X.Y.Z Development project for development toward the next official release

Define Related Issue(s)

Consider the impact to the other METplus components.

[x] METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy No impacts.

Bugfix Checklist

See the METplus Workflow for details.

[ ] Complete the issue definition above, including the Time Estimate and Funding Source.
[ ] Fork this repository or create a branch of main_\. Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
[ ] Fix the bug and test your changes.
[ ] Add/update log messages for easier debugging.
[ ] Add/update unit tests.
[ ] Add/update documentation.
[ ] Add any new Python packages to the METplus Components Python Requirements table.
[ ] Push local changes to GitHub.
[ ] Submit a pull request to merge into main_\. Pull request: bugfix <Issue Number> main_<Version> <Description>
[ ] Define the pull request metadata, as permissions allow. Select: Reviewer(s) and Development issue Select: Milestone as the next bugfix version Select: Coordinated METplus-X.Y Support project for support of the current coordinated release
[ ] Iterate until the reviewer(s) accept and merge your changes.
[ ] Delete your fork or branch.
[ ] Complete the steps above to fix the bug on the develop branch. Branch name: bugfix_<Issue Number>_develop_<Description> Pull request: bugfix <Issue Number> develop <Description> Select: Reviewer(s) and Development issue Select: Milestone as the next official version Select: METplus-Wrappers-X.Y.Z Development project for development toward the next official release
[ ] Close this issue.

j-opatz commented 5 days ago

I've tweaked a few things to the existing configuration file, including adding climo field information and updating the configuration file arguments requesting line types. However, the use case still seems to be unable to process the incoming climatology field. The ANOM_CORR and RMSFA variable fields are empty.

Additional tests of bogus data as climo data also ended in no climatology being read in.

Next step will be to build a simplified version of this configuration file, only allowing 1 run (rather than the 6 months of lead time) and see if something comes across in the ANOM_CORR and RMFSA fields. If it continues to prove elusive or consumes too much analysis time this use case may be removed from the METplus repository, with the primary reason being if the climatology function does not work, then the purpose of the use case is null.

j-opatz commented 1 day ago

Continued testing today. It seems that there may be a bigger issue at hand: when I did a simple GridStat run with nothing additional other than fcst, obs, and climo mean information, the nc_flag option does not put out a climo field entry. There's nothing in the log saying the read in was unsuccessful, though. It seems like a recent change may have stopped METplus from creating climo-related output.

As a second check, I will run a direct call to Grid-Stat via MET using the exact same set up (files, fields, etc) and see if nc_flags will produce a climo field. If not, additional eyes may be needed on this issue.

dtcenter / METplus