search

Striveworks / valor

Valor is a centralized evaluation store which makes it easy to measure, explore, and rank model performance.
https://striveworks.github.io/valor/
Other
38 stars 4 forks source link

BUG: ROCAUC is not supported for Binary Classifications #780

Open czaloom opened 1 month ago

czaloom commented 1 month ago

valor version checks

Reproducible Example

See sklearn's handling of single class data being passed to `roc_auc_score`

Issue Description

The lite ROCAUC implementation is intended for multiclass data and should not be used with binary classification datasets.

Expected Behavior

We should raise a warning or not return ROCAUC if an invalid input is passed.