@jtjohnston brought up some concerns about how the ConfusionMatrix metric details hallucinations separately. The primary concern is that it is not fully clear how hallucinations relate to overall false-positives and how it relates to the confusion matrix value.
Feature Description
Option 1
Move hallucinations and missing_predictions into the confusion matrix by designating a null or background label.
Option 2
Rename hallucinations to unmatched_predictions
Rename missing_predictions to unmatched_ground_truths
Feature Type
[ ] Adding new functionality to valor
[X] Changing existing functionality in valor
[ ] Removing existing functionality in valor
Problem Description
@jtjohnston brought up some concerns about how the
ConfusionMatrixmetric detailshallucinationsseparately. The primary concern is that it is not fully clear how hallucinations relate to overall false-positives and how it relates to the confusion matrix value.Feature Description
Option 1
hallucinationsandmissing_predictionsinto the confusion matrix by designating anullorbackgroundlabel.Option 2
hallucinationstounmatched_predictionsmissing_predictionstounmatched_ground_truthsAdditional Context
No response