COCO-fashion evaluation code for oriented objects

[qjadud1994]

Hello.

I want to evaluate the performance of an oriented object in COCO fashion.

I heard you evaluate the performance of the SKU110K-R dataset in the same way as COCO.

Did you calculate rotated IoU for performance evaluation?

Also, could you share the COCO-style evaluation code for rotated objects?

I look forward to your answer.

[Panxjia]

We have uploaded two necessary tools for the evaluation of rotated objects.

[qjadud1994]

Thank you very much for your quick reply.

It will be of great help to me ^^.

[qjadud1994]

Could you give me guidelines for using ro_cocoapi?

According to the annotation, I figured out that the 'rbbox' is [center_x, center_y, width, height, angle in radian].

And I use ground truth labels as prediction results, I run the code below, hoping for 100% mAP to come out.

import os
import json

from pycocotools_ro.coco import COCO
from pycocotools_ro.cocoeval import COCOeval

gt_anno_path = os.path.join('/data/DB', 'SKU110K', 'annotations', 'sku110k-r_val.json')
gt_json_file = open(gt_anno_path)
gt_json_data = json.load(gt_json_file)

predictions = []

for gt in gt_json_data["annotations"]:
    predictions.append(
        {
            'image_id': int(gt['image_id']),
            'category_id': 1,
            'segmentation': gt['segmentation'],
            'rbbox' : gt['rbbox'],
            'score': 0.9,
        }
    )

coco_gt = COCO(gt_anno_path)
coco_dt = coco_gt.loadRes(predictions)
coco_eval = COCOeval(coco_gt, coco_dt, 'rbbox')

coco_eval.evaluate()
coco_eval.accumulate()
coco_eval.summarize()

However, I got a result of 64.4%mAP.

How can I fix it? Please tell me about the correct prediction format and how to use the ro_cocoapi.

Thanks.

[Panxjia]

@qjadud1994 Sorry for the late reply. I can reproduce the issue. The key is the max_obj_number for each image. You can try to increase the number as follows,

coco_eval = COCOeval(self.coco, coco_dets, "rbbox")
coco_eval.params.maxDets = [1, 10, 300]

In the sku110k paper, the author set the max_det_number to 300. However, it still can't reach 100% mAP(97% when the max_det=300).

[qjadud1994]

Thanks for answering.

After correcting what you said, I got 97% results.