Open feiyuhuahuo opened 4 years ago
Those gts whose 'iscrowd'
is 1 are included in the calculation.
The cocoEval.evalImgs
contains the matching infomation between gts
and dts
, which provides necessary information for computing mAP. In many value of 'gtIds'
and 'dtMatches'
, there are gts
whose 'iscrowd'
is 1.
Notably, the IoU computation involved gts
whose 'iscrowd'
is 1 is different from the standard, as described in the comment of the file mask.py
:
"Finally, a note about the intersection over union (iou) computation. The standard iou of a ground truth (gt) and detected (dt) object is iou(gt,dt) = area(intersect(gt,dt)) / area(union(gt,dt)). For "crowd" regions, we use a modified criteria. If a gt object is marked as "iscrowd", we allow a dt to match any subregion of the gt. Choosing gt' in the crowd gt that best matches the dt can be done using gt'=intersect(dt,gt). Since by definition union(gt',dt)=dt, computing iou(gt,dt,iscrowd) = iou(gt',dt) = area(intersect(gt,dt)) / area(dt). For crowd gt regions we use this modified criteria above for the iou."
I am doing some tests on object detection tasks. I find that if I do not append the crowd ground-trouth objects to
gts
, the final result is slightly different, 34.6 mAP vs 34.3 mAP. I change this line: https://github.com/cocodataset/cocoapi/blob/8c9bcc3cf640524c4c20a9c40e89cb6a2f2fa0e9/PythonAPI/pycocotools/cocoeval.py#L115 to:So I wonder whether a crowd
gt
is involved in the calculation? @pdollar @tylin