Striveworks / valor

Valor is a centralized evaluation store which makes it easy to measure, explore, and rank model performance.
https://striveworks.github.io/valor/
Other
38 stars 4 forks source link

ENH: Data filtering should affect returned metrics. #766

Closed czaloom closed 2 months ago

czaloom commented 2 months ago

Feature Type

Problem Description

Valor Lite retains the metadata for the complete dataset / model. While not incorrect, this leads to null metrics being returned when all groundtruths of a particular label are filtered out.

Feature Description

Instead of returning null metrics, return nothing.

Additional Context

No response