search

NOAA-OWP / wres

Code and scripts for the Water Resources Evaluation Service
Other
2 stars 1 forks source link

As a user, I want additional metrics for extreme/rare events #133

Open epag opened 3 weeks ago

epag commented 3 weeks ago

Author Name: James (James) Original Redmine Issue: 120818, https://vlab.noaa.gov/redmine/issues/120818 Original Date: 2023-09-22

Given a need to measure forecast quality for rare or extreme events When I consider the metrics available in the WRES Then I would like to see more metrics with good statistical properties for extreme events

epag commented 3 weeks ago

Original Redmine Comment Author Name: James (James) Original Date: 2023-09-22T12:14:20Z

There is increasing demand for this from HEFS and other teams. Many conventional metrics tend to trivial values (e.g., 0) as the climatological probability of the event becomes rarer.

It would be good to have one or two metrics with good statistical properties for rare events for each data type, notably continuous numerical (ensemble and single-valued) and categorical (single-valued and probabilistic).

Examples include the threshold-weighted CRPS (with various weight function possibilities) for continuous numerical ensemble predictions and SEDI for single-valued categorical (dichotomous) predictions.

We can add a checklist in due course.