uncharted-distil / distil-auto-ml

Distil Automated Machine Learning Server
Apache License 2.0
2 stars 1 forks source link

Properly handle confidences for all metrics #230

Closed cdbethune closed 4 years ago

cdbethune commented 4 years ago

We handle the presence of confidence values during scoring properly when computing a roc-auc score, but for other metrics we don't properly account for the multiple predictions per d3mIndex, leading to incorrect scores.