IOU error

[cvchanghao]

Thank you so much for sharing! But maybe I found an error in utils.py. When computing IOU, you filtered the pred where mask == 0. It will make the IOU higher.（line 47） Is it correct?

[chenxi116]

So there are some datasets that use class 255 to label pixels that are ambiguous. These pixels should not count towards IOU calculation.

In utils.py I first made sure they are uint8, and then add 1 to the tensor, which will make these 255 to 0. Then these pixels with label 0 (originally labeled 255) are filtered out.

Let me know if this explanation aligns with the code?

[cvchanghao]

Yes your explanation aligns with the code. But I think we should not do like this. for example, If the pred has some non-zero prediction but you filtered out, then the area_union will be smaller. which will make the iou larger.
Sorry for my poor English. 举个例子，如果pred里预测的label为1的像素位置，有一些在mask里是255的，那么这部分错误预测就会被过滤掉。在计算IOU时，分母就变小了，IOU自然就变大了。

[chenxi116]

I see your concern, but I think another good property of the metric would be: 0.0 for the worst algorithm in the world, and 1.0 for the best algorithm in the world. If I understand your proposal correctly, for datasets with any ambiguous pixel, this metric can never reach 1.0? To me this is not ideal.

Another important aspect is to align with prior implementations, so that the numbers are directly comparable. I believe I checked this implementation against PASCAL calculation and ADE20K calculation.

[cvchanghao]

Thank you, now i got it