search

dbolya / tide

A General Toolbox for Identifying Object Detection Errors
https://dbolya.github.io/tide
MIT License
706 stars 115 forks source link

Encountered a problem with PASCAL VOC #29

Closed LiewFeng closed 3 years ago

LiewFeng commented 3 years ago

Hi, @dbolya , I'm interested in this work, but encountered a problem on Pascal VOC dataset, a very low mAP. 71.1 in the mmdetection v.s. 5.7 in tide. I tried to find the reason for several days, but failed. Could you kindly give some suggestions? Thanks a lot!

Code related to tide is the following, gt = datasets.Pascal(path='pascal_test2007.json') pred = datasets.COCOResult(path='pre.json.bbox.json') tide = TIDE() tide.evaluate_range(pred, gt, mode=TIDE.BOX ) tide.summarize()

I convert the detection results to COCO json style with the following code, image

The results are as follows, image

dbolya commented 3 years ago

Verify that the xywh of the bounding boxes are in absolute coordinates, not relative coordinates. I.e., the x,y,w,h should be in number of pixels, not a proportion of image size.

LiewFeng commented 3 years ago

I check the xywh in the predicted json. They are all in absolute coordinates, larger than 1.

dbolya commented 3 years ago

Are you sure that your image ids match up with those found in the annotations file?

LiewFeng commented 3 years ago

It seems matched. I conduct an experiment to veryfy this. For the same detection results, I set the img_id as a random int with the following code, img_id = np.random.randint(1,len(self)) I run 5 times and get mAP as 0.19, 0.17, 0.25, 0.31, 0.19, which if lower than the mAP of original img_id as 5.7. img_id = self.img_ids[idx]

LiewFeng commented 3 years ago

Fixed. Exchanging the position of pred and gt works.