search

plemeri / InSPyReNet

Official PyTorch implementation of Revisiting Image Pyramid Structure for High Resolution Salient Object Detection (ACCV 2022)
MIT License
497 stars 73 forks source link

What the T stands for in TIou? #26

Closed sanghyun0927 closed 1 year ago

sanghyun0927 commented 1 year ago
  1. Can you explain the difference between bIoU and tIoU, or post a link to a reference about tIoU?
sanghyun0927 commented 1 year ago

chatGPT says "In the context of the provided code, the 't' in TIoU stands for "thin," as the TIoU class calculates a thin (boundary-based) version of the Intersection over Union metric, similar to the BIoU class.". Think he's right.

plemeri commented 1 year ago

Hi, TIoU was something I tried to make my own boundary related evaluation metric which I failed. The evaluation script in this repository is identical to the BIoU.

Sorry for the misleading content 😢

sanghyun0927 commented 1 year ago

Oh, my.. got it!

sanghyun0927 commented 1 year ago

I am becoming more inclined to trust avgTIou over mBA in evaluating test scores. I'd like to continue to use avgTIou as an evaluation metric, but I'm curious what the author thinks.

plemeri commented 1 year ago

As I mentioned before, TIoU is identical to the BIoU only for this repository, and I was also worried about considering the right boundary metric. In my opinion, BDE, which was used in PGNet, was not a good option for boundary metric since it wasn't quite stable enough for measuring the performance and there wasn't any accessible official evaluation script. On the other hand, BIoU and mBA were producing quite stable values and we could easily find official evaluation script, so I chose one of them.

Long story short, I think BIoU and mBA are both okay to use. Maybe BIoU is more appropriate measure for pixel-wise prediction since they are both identical in terms of measuring only around the boundary area since BIoU measures IoU around the edges (which is more common in this field) while mBA measures accuracy only.

Thanks for sharing your thought.