uclamii / model_tuner

A library to tune the hyperparameters of common ML models. Supports calibration and custom pipelines.
Apache License 2.0
3 stars 0 forks source link

Confusion Matrix Printout - Misaligned Labels #32

Closed lshpaner closed 1 month ago

lshpaner commented 1 month ago

Description:

The confusion matrix printout has misaligned "Pos" and "Neg" labels under the "Predicted" and "Actual" sections. This leads to incorrect associations between the labels and the corresponding confusion matrix values.

The return_metrics generates the following confusion matrix using the AIDS Clinical Trials Notebook as an example:

Confusion matrix on set provided: 
--------------------------------------------------------------------------------
          Predicted:
             Pos   Neg
--------------------------------------------------------------------------------
Actual: Pos 304 (tp)   20 (fn)
        Neg  40 (fp)   64 (tn)
--------------------------------------------------------------------------------

              precision    recall  f1-score   support

           0       0.88      0.94      0.91       324
           1       0.76      0.62      0.68       104

    accuracy                           0.86       428
   macro avg       0.82      0.78      0.80       428
weighted avg       0.85      0.86      0.85       428

Let us observe that the "Pos" and "Neg" labels do not match the actual confusion matrix values.

The true and correct confusion matrix output should be as follows:

Confusion matrix on set provided: 
--------------------------------------------------------------------------------
          Predicted:
             Pos   Neg
--------------------------------------------------------------------------------
Actual: Pos  64 (tp)   40 (fn)
        Neg  20 (fp)  304 (tn)