search

bes-dev / mean_average_precision

Mean Average Precision for Object Detection
MIT License
193 stars 32 forks source link

Differences in COCO and VOC evaluation protocol #1

Open vadimadr opened 4 years ago

vadimadr commented 4 years ago

COCO and VOC use different protocols for assigning tp / fp labels for predicted boxes.

VOC uses "greedy" strategy, i.e. finds best match (using IoU criteria) for current pred box and if it is already matched it marks the current pred box as false: https://github.com/weiliu89/VOCdevkit/blob/master/VOCcode/VOCevaldet.m#L94

While in COCO the search continues if the current best match already matched: https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/cocoeval.py#L280 Also, there are other categories for gt boxes: "crowd" (which look like VOC's "difficult") and "ignore". There might be other differences as well.

Your code implements only VOC-style evaluation, while README suggests to use it for both flavors.

bes-dev commented 4 years ago

There is implemented COCO evaluation protocol in actual release version. https://github.com/bes-dev/mean_average_precision/releases/tag/0.0.2.1