search

dtcenter / MET

Model Evaluation Tools
https://dtcenter.org/community-code/model-evaluation-tools-met
Apache License 2.0
77 stars 24 forks source link

Improve the uniqueness key used for the duplicate_flag logic in Point-Stat and Ensemble-Stat. #226

Open dwfncar opened 11 years ago

dwfncar commented 11 years ago

When running Point-Stat with the duplicate_flag set to SINGLE and verifying 2-meter temperature, Michelle Harrold found an issue. Point-Stat found two matches for the KDEN station id, when we would have only expected one. This task is to investigate the source of the two matches and resolve it. [MET-226] created by johnhg

dwfncar commented 11 years ago

Upon close inspection, we found that the two matching observations for KDEN had very slightly different pressure level values. The pressure level is included in the uniqueness key, so those observations were treated as being different. However, since this is 2-meter temperature, we'd really like them to be considered the same.


The fix was to update the logic in the PairDataPoint and PairDataEnsemble classes. When verifying a forecast field with a vertical level type against an observation whose message type is ADPSFC or SFCSHP, set the observation level value to bad data.


That change results in a single match for KDEN during that time period and doesn't break the regression test done during the nightly build.


Should add information about this to the release notes. by johnhg

dwfncar commented 11 years ago

We discussed this issue and my resolution for it at the MET development meeting on 3/21. Setting the pressure level value to bad data is not desirable behavior. It would be preferable to find another solution that does not have that effect. Investigate alternative solutions for this problem.


Also, enhance the unit tests for Point-Stat to ensure that they include verification of 2-m temperature vs the ADPSFC message type. I would have expected to see differences in the MPR output of the Point-Stat tool after this change. by johnhg