search

dtcenter / MET

Model Evaluation Tools
https://dtcenter.org/community-code/model-evaluation-tools-met
Apache License 2.0
77 stars 24 forks source link

Consider enhancing tc_stat to refine event equalziation logic using percentages. #605

Open dwfncar opened 8 years ago

dwfncar commented 8 years ago

Andy Penny from GFDL requested via met_help that the logic in tc_stat be refined to mimic what NHC does in it's end-of-season reports:
   https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=76595


Here's his request:
Would it be difficult to add an option to tc_stat such that models are only
included in a homogeneous verification if they are present for a certain
percentage of the total number of forecasts? For example, NHC typically
will only include a model in the homogeneous verification if it's present
70% of the time at 0 h and 67% of the time at 48 h.


Sorry for hijacking this ticket for another issue, but I wanted to get your
thoughts on whether this might be something that could be added in a future
release? We're currently doing this check outside of MET-TC, but it might
be a nice feature.


...


Yes, actual percentages are used at NHC for the end of season/storm
verification reports. A model is included in the homogeneous verification
only if it was present for 2/3 of the forecasts at both 48 h and 120 h.
This isn't a hard fast rule, as this constraint can be relaxed for
shorter-lived systems with fewer verifiable forecasts.

[MET-605] created by johnhg

dwfncar commented 8 years ago

Ask Paul Kucera if HFIP can pay for this. by johnhg