Striveworks / valor

Valor is a centralized evaluation store which makes it easy to measure, explore, and rank model performance.
https://striveworks.github.io/valor/
Other
38 stars 4 forks source link

Semantic Segmentation Bugfix #815

Closed czaloom closed 2 weeks ago

czaloom commented 2 weeks ago

Issue

Summing was performed on the wrong axes for counting unmatched pixels. This did not raise issues as it only occurred in certain edge cases.

Testing

Added an edge case test. It checks the output of an intermediate confusion matrix.

The structure of the intermediate is as follows.

Position (0, 0) is count of background pixels. Slice (0, 1:) are unmatched prediction counts. Slice (1:, 0) are unmatched ground truth counts.

# before fix
[[ 0 -1 -1  3 -1]
 [ 1  0  0  1  0]
 [ 1  0  0  1  0]
 [-3  0  0  1  0]
 [ 1  0  0  1  0]]

# after fix
[[0 0 0 0 0]
 [0 0 0 1 0]
 [0 0 0 1 0]
 [0 0 0 1 0]
 [0 0 0 1 0]]