search

CSAILVision / semantic-segmentation-pytorch

Pytorch implementation for Semantic Segmentation/Scene Parsing on MIT ADE20K dataset
http://sceneparsing.csail.mit.edu/
BSD 3-Clause "New" or "Revised" License
4.91k stars 1.1k forks source link

Why mIoU uses sum(intersect)/sum(union) instead of mean over IoU? #271

Open hubert0527 opened 2 years ago

hubert0527 commented 2 years ago

I found the mIoU evaluation codes here: https://github.com/CSAILVision/semantic-segmentation-pytorch/blob/8f27c9b97d2ca7c6e05333d5766d144bf7d8c31b/eval.py#L98

I am wondering what is the motivation of using sum(intersect)/sum(union) instead of the mean over sample-wise intersect / union? The former one seems to disregard the testing samples are iid.

Thanks!

apple2373 commented 9 months ago

This appears to be a common issue in semantic segmentation. Some use global IoU, while others use the average per-image IoU.

https://github.com/IvLabs/stagewise-knowledge-distillation/issues/12#issuecomment-650696538 https://stats.stackexchange.com/questions/554724/is-there-an-official-procedure-to-compute-miou-mean-intersection-over-union

I've noticed others raising similar queries, and unfortunately, there's no universally accepted method. I think it's crucial to specify which approach you're using, and to ensure that you compare the same mIoU when publishing a paper.