frazane / scoringrules

Scoring rules for probabilistic forecast evaluation and optimization.
https://frazane.github.io/scoringrules/
Apache License 2.0
41 stars 6 forks source link

Flexible NaN-handling #34

Open frazane opened 3 months ago

frazane commented 3 months ago

For ensemble-based scoring rules, we need flexible handling of missing values in ensemble members.

Currently, ensemble-based metrics such as the CRPS return NaN if there is one or more NaNs in the ensemble members. It may be the case that users have an ensemble with a few NaNs (e.g. with lagged ensembles you have NaNs for some timestamps) but still want to get a valid score.

Also ensures consistency with https://github.com/nci/scores/blob/develop/src/scores/probability/crps_impl.py