Will the ['iscrowd'] attribution influence the final result?

[feiyuhuahuo]

I am doing some tests on object detection tasks. I find that if I do not append the crowd ground-trouth objects to gts, the final result is slightly different, 34.6 mAP vs 34.3 mAP. I change this line: https://github.com/cocodataset/cocoapi/blob/8c9bcc3cf640524c4c20a9c40e89cb6a2f2fa0e9/PythonAPI/pycocotools/cocoeval.py#L115 to:

So I wonder whether a crowd gt is involved in the calculation? @pdollar @tylin

[jxl152]

Those gts whose 'iscrowd' is 1 are included in the calculation.

The cocoEval.evalImgs contains the matching infomation between gts and dts, which provides necessary information for computing mAP. In many value of 'gtIds' and 'dtMatches', there are gts whose 'iscrowd' is 1.

Notably, the IoU computation involved gts whose 'iscrowd' is 1 is different from the standard, as described in the comment of the file mask.py:

"Finally, a note about the intersection over union (iou) computation. The standard iou of a ground truth (gt) and detected (dt) object is iou(gt,dt) = area(intersect(gt,dt)) / area(union(gt,dt)). For "crowd" regions, we use a modified criteria. If a gt object is marked as "iscrowd", we allow a dt to match any subregion of the gt. Choosing gt' in the crowd gt that best matches the dt can be done using gt'=intersect(dt,gt). Since by definition union(gt',dt)=dt, computing iou(gt,dt,iscrowd) = iou(gt',dt) = area(intersect(gt,dt)) / area(dt). For crowd gt regions we use this modified criteria above for the iou."