nci / scores

scores: Metrics for the verification, evaluation and optimisation of forecasts, predictions or models.
https://scores.readthedocs.io/
Apache License 2.0
82 stars 25 forks source link

703 warn user when pearsonr returns nan + a flag enable or disable the warning message (default is disable) #776

Open JinghanFu opened 3 days ago

JinghanFu commented 3 days ago

Please work through the following checklists. Delete anything that isn't relevant.

Development for new xarray-based metrics

Docstrings

Testing of new xarray-based metrics

Tutorial notebook

Documentation

tennlee commented 3 days ago

@nicholasloveday I think this PR is interesting from the perspective of what kinds of conditions should raise a user warning. Perhaps for all our methods, anything which is all NaN should raise a user warning? Correct setting of NaNs is intended behaviour, so doesn't indicate a program failure. On the other hand, something being entirely NaN could be an indication of a data issue or user error, and a warning could be useful. We should probably consider adopting something consistent across the codebase. Take a look at what @JinghanFu has done in this example and have a think. This work was done because @durgals mentioned it following his KGE work as being useful. @durgals you might also like to make a comment about what kinds of user warning should perhaps be raised across all our methods. Thanks @JinghanFu for putting together a concrete example for us to discuss.