search

dbolya / tide

A General Toolbox for Identifying Object Detection Errors
https://dbolya.github.io/tide
MIT License
702 stars 115 forks source link

Dupe detection seems to be working incorrectly #6

Closed kirillfish closed 4 years ago

kirillfish commented 4 years ago

When testing an object detector on my custom dataset, I found out that the most prevalent error is duplicate bboxes. It is clearly seen when I visualize detected bboxes. But TIDE doesn't recognize that and always reports tiny, almost zero amount of duplicates.

I also noticed that there is no example in your paper or notebooks where Dupe category would be significant fraction of all errors, which looks dubious. Are you sure there is no bug in here? For example, what is ex.gt_used_cls and is it defined properly?

https://github.com/dbolya/tide/blob/master/tidecv/quantify.py#L251

kirillfish commented 4 years ago

Now I see that it detects duplicates, but for some reason doesn't include into the summary

kirillfish commented 4 years ago

I changed fix method for DuplicateError:

def fix(self):
    return self.suppressor['class'], -1

It changed the way summarizer processes fixed errors, and now it seems to report them correctly