map score of zero

[traderandreas]

I have created a results file for bounding box detections in the format outlined by the coco api for the 2014 dataset. When I evaluate my results the map scores are zero? The images show perfect bounding boxes localising the object. The image ids match the GT and results DT. What could be the issue?

[harrisonford]

Hi! I actually had a similar problem using my own dataset and mimic the COCO annotation file for the keypoint evaluation.

I found out that the "bbox" parameter and the "area" are not related such as "width x height" because "area" is actually the real area of the segmentation not the bounding box.

For example:

bbox = [0.84 114.71, 53.54, 257.7]; area = 8703.874

If area was wxh it should be 13797.258. So with overvalued "area" parameters when COCOapi computes the IoU values they could be an order of magnitude smaller, and since IoU threshold starts from 0.5 all my IoU's are cut off.

In simple words check if the "area" parameter in the annotations is similar to COCO's, which is not width x height of the bbox. Not sure if your case is similar to mine but it's worth the check.

[mmanoharahere]

Hi @harrisonford , I have run into the same problem. How did you solve it? Please let me know. Thank you.

[Roujack]

I think that the area in annotation and normalization constant for each keypoint cause this problem. According to the evaluation metric OKS (https://cocodataset.org/#keypoints-eval), to get a reasonable map result, you need to reset the per-keypoint constant ki, which means that you have to label your test set twice and calculate the ki through standard deviation σi. I don't relabel my test set since it costs much time. I modify the area by multiply a constant (50) instead to satisfy the similarity threshold from 0.5 to 0.95.