search

XiaohangZhan / deocclusion

Code for our CVPR 2020 work.
Apache License 2.0
795 stars 104 forks source link

Question about your metric computation #10

Closed sydney0zq closed 4 years ago

sydney0zq commented 4 years ago

Hi Xiaohang Zhan,

Your framework seems to be elegant and easy to use. And the work you proposed is inspiring. However, I think there is one unreasonable computation on your metric (IOU).

https://github.com/XiaohangZhan/deocclusion/blob/c8439ea5c119ec1d2cfab5f8ba4694de88e5b0b4/tools/test.py#L195

As we can see, you sum all foreground pixels and all background pixels together of the whole dataset, and then use the total pixel number to get the final IOU result. As far as my experience, it is not the common style to calculate IOU. Although you also use the computation way for other methods, it is fair in your paper.

Could you please clarify the computation style? I think it significantly enlarges the metric difference between Method Raw and Method PCNet-M.

Regards, Qiang Zhou

XiaohangZhan commented 4 years ago

I followed the style of mIoU in semantic segmentation that accumulates intersections and unions across the dataset first and then makes the division. Refer to https://github.com/CSAILVision/semantic-segmentation-pytorch/blob/master/eval.py