In my "are my labels correct!?"-research I've often relied on this function:
def correct_class_confidence(X, y, mod):
"""
Gives the predicted confidence (or proba) associated
with the correct label `y` from a given model.
"""
probas = mod.predict_proba(X)
values = []
for i, proba in enumerate(probas):
proba_dict = {mod.classes_[j]: v for j, v in enumerate(proba)}
values.append(proba_dict[y[i]])
return values
You can take the average/std of these numbers and this says something about confidence and variability, see here. Would be useful to have around in a library.
koaning
commented
2 years ago
In my "are my labels correct!?"-research I've often relied on this function:
You can take the average/std of these numbers and this says something about confidence and variability, see here. Would be useful to have around in a library.