dtcenter / MET

Model Evaluation Tools
https://dtcenter.org/community-code/model-evaluation-tools-met
Apache License 2.0
74 stars 22 forks source link

Bugfix: Fix inconsistent handling of point observation valid times processed through Python embedding #2897

Closed JohnHalleyGotway closed 1 month ago

JohnHalleyGotway commented 1 month ago

Describe the Problem

The METplus Point-Stat TCI Use Case recently produced unexpected differences when run via GitHub Actions. Careful investigation by @DanielAdriaansen (documented here) showed the following:

  1. Re-create the dtcenter/METbaseimage v3.2 release triggered these differences.
  2. The updated image lead to the Python glob serving up files to a MET Python embedding script in a different order.
  3. The different order of input files lead to a different order of point observations being served up in memory to Point-Stat.
  4. A bug in how valid time strings are stored for Python embedding of point observations produced different results from Point-Stat. Specifically, the observation are filtered by their valid time in an inconsistent way.

Expected Behavior

The order of point observations served up via Python embedding to Point-Stat should have no impact on filtering by valid time.

To Reproduce

Describe the steps to reproduce the behavior:

  1. Download this sample data file: issue_2897_data.tar.gz
  2. Unpack the file and update the MET_BIN setting in run_ps.sh to point to a local MET binary directory.
  3. Execute run_ps.sh.
  4. The following command:
    for OBS_SID in `cat out_test2/point_stat_000000L_19790101_000000V_mpr.txt | awk '{print $27}'`; do
    grep $OBS_SID test1.csv;
    grep $OBS_SID test2.csv;
    echo "$OBS_SID test1: `grep $OBS_SID out_test1/point_stat_000000L_19790101_000000V_mpr.txt | awk '{print $8, $28, $29}'`";
    echo "$OBS_SID test2: `grep $OBS_SID out_test2/point_stat_000000L_19790101_000000V_mpr.txt | awk '{print $8, $28, $29}'`";
    echo;
    done
  5. Note the following timestamp mismatch:
    ADPSFC CH-Oe1 20020128_000000 47.2858 7.7319 10 TCI 10 0 NA 1.1331664358420377
    CH-Oe1 test1: 20020128_000000 47.2858 7.7319
    CH-Oe1 test2: 20051201_000000 47.2858 7.7319

    In test2, the timestamp for CH-Oe1 is WRONG.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

Labels

Milestone and Projects

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.