search

nci / scores

scores: Metrics for the verification, evaluation and optimisation of forecasts, predictions or models.
https://scores.readthedocs.io/
Apache License 2.0
75 stars 23 forks source link

[JOSS] Implementation of weights is occasionally unclear #578

Open calebweinreb opened 4 months ago

calebweinreb commented 4 months ago

Re JOSS review openjournals/joss-reviews#6889

A big strength of scores is the ability to incorporate weights into many of the metrics. But how exactly these weights are incorporated is often unclear. Though the application of weights is obvious in come cases (e.g. RMSE), I don't have any intuition for how weights be used roc_curve_data, for example. In these ambiguous cases the docstring should be updated to provide a clearer explanation.

tennlee commented 4 months ago

That makes sense. It will take some time, but I will work through each score/metric and check if there is an example of the weights presented in each notebook as well as checking the docstrings to provide an indication. If you notice any others you think are particularly ambiguous, feel free to drop a note here.