Correct Class Confidence Implementation

[koaning]

In my "are my labels correct!?"-research I've often relied on this function:

def correct_class_confidence(X, y, mod):
    """
    Gives the predicted confidence (or proba) associated
    with the correct label `y` from a given model.
    """
    probas = mod.predict_proba(X)
    values = []
    for i, proba in enumerate(probas):
        proba_dict = {mod.classes_[j]: v for j, v in enumerate(proba)}
        values.append(proba_dict[y[i]])
    return values

You can take the average/std of these numbers and this says something about confidence and variability, see here. Would be useful to have around in a library.

[koaning]

This has moved to doubtlab.