rotated object detection

[ZhenboZhao77]

Dear Author,

I have been deeply inspired after reading your MoCaE paper and would like to congratulate you on achieving state-of-the-art results on the DOTA dataset. I am interested in obtaining the code related to the application of MoCaE in rotated object detection. Could you please advise on how I might access it? I would be very grateful for your response.

[kemaloksuz]

We combined the detectors from different repositories for rotated bounding box detection. That's why, unfortunately, incorporating all the repositories together in a single repository is quite difficult and it is unlikely that we will add that part of the code into this repository in the near future.

Having said that, we can try to help you in reproducing our results using this repository if you are willing to do so. If so, below are the main steps to be followed to reproduce the results:

1. First detector we used is RTMDet. This is the repo and the detector: You should use the one with 81.33 AP. You should generate an output COCO-style json file on the val set with this repo. As an example for COCO-style each detection should be something like this: {'image_id': 25, 'score': 0.97, 'bbox': [385.79, 1783.79, 407.02, 1137.92, 89.9, 1127.49, 68.66, 1773.36], 'category_id': 4}

2. Please follow Step 1 on the second detector we used, that is LSKN. You should now use this repo: https://github.com/zcablii/LSKNet

3. You should obtain a calibrator for each of these models using this script. To do so, you should modify few lines (237-240 in particular). 237 and 238 should be validation set ground truth file in coco style. 239 and 240 should be the detection file you have. So, 239 and 240 should be changed across runs. At the end of the day, you should get two calibrators (one for each model). There is one more line that you need to modify here. Instead of standard IoU, rotated bbox iou should be computed in Line 38. I imported box_iou_quadri for that, that is from mmcv.ops import box_iou_quadri and then replace assign_post function with the function below:

def assign_rotated(ann_dict, det_bboxes, det_score, det_label, dataset_classes):
    num_classes = len(dataset_classes)
    ious = np.zeros([det_bboxes.shape[0]])
    ## Assign
    for k, v in ann_dict.items():
        # Convert to numpy and reshape
        gt_boxes = np.array(v).reshape(-1, 8)
        rel_idx = (det_label==k).nonzero()[0]
        ious_cl = (box_iou_quadri(torch.from_numpy(gt_boxes).float(), torch.from_numpy(det_bboxes[rel_idx]).float())).numpy()
        ious[rel_idx] = np.max(ious_cl, axis=0)
    return ious

4. You should now obtain the detections on the test set for each detector.

5. You need to combine the detections of the test set while calibrating them using the calibrators obtained on step 3. Something like this configuration file should work. One additional change should be using NMS for rotated bounding boxes here.

6. You have the combined detections in COCO-format now. Using DOTA toolkit you can convert it to DOTA format and submit to the eval server.

This is some work but it should help if you need to reproduce our results. Please feel free to let us know if you have questions.

[July-1024]

We combined the detectors from different repositories for rotated bounding box detection. That's why, unfortunately, incorporating all the repositories together in a single repository is quite difficult and it is unlikely that we will add that part of the code into this repository in the near future.

Having said that, we can try to help you in reproducing our results using this repository if you are willing to do so. If so, below are the main steps to be followed to reproduce the results:

1. First detector we used is RTMDet. This is the repo and the detector: You should use the one with 81.33 AP. You should generate an output COCO-style json file on the val set with this repo. As an example for COCO-style each detection should be something like this: {'image_id': 25, 'score': 0.97, 'bbox': [385.79, 1783.79, 407.02, 1137.92, 89.9, 1127.49, 68.66, 1773.36], 'category_id': 4}
2. Please follow Step 1 on the second detector we used, that is LSKN. You should now use this repo: https://github.com/zcablii/LSKNet
3. You should obtain a calibrator for each of these models using this script. To do so, you should modify few lines (237-240 in particular). 237 and 238 should be validation set ground truth file in coco style. 239 and 240 should be the detection file you have. So, 239 and 240 should be changed across runs. At the end of the day, you should get two calibrators (one for each model). There is one more line that you need to modify here. Instead of standard IoU, rotated bbox iou should be computed in Line 38. I imported box_iou_quadri for that, that is and then replace assign_post function with the function below:from mmcv.ops import box_iou_quadri

def assign_rotated(ann_dict, det_bboxes, det_score, det_label, dataset_classes):
    num_classes = len(dataset_classes)
    ious = np.zeros([det_bboxes.shape[0]])
    ## Assign
    for k, v in ann_dict.items():
        # Convert to numpy and reshape
        gt_boxes = np.array(v).reshape(-1, 8)
        rel_idx = (det_label==k).nonzero()[0]
        ious_cl = (box_iou_quadri(torch.from_numpy(gt_boxes).float(), torch.from_numpy(det_bboxes[rel_idx]).float())).numpy()
        ious[rel_idx] = np.max(ious_cl, axis=0)
    return ious

4. You should now obtain the detections on the test set for each detector.
5. You need to combine the detections of the test set while calibrating them using the calibrators obtained on step 3. Something like this configuration file should work. One additional change should be using NMS for rotated bounding boxes here.
6. You have the combined detections in COCO-format now. Using DOTA toolkit you can convert it to DOTA format and submit to the eval server.

This is some work but it should help if you need to reproduce our results. Please feel free to let us know if you have questions.

Hi，Can you give a little more detailed steps? How do I generate an output coco-style json file in the first step? In addition, mmrotate1.x does not integrate LSKNet, do I need to add it manually?I'm stuck at the first step at the moment.Looking forward to your prompt answer.

[selimkuzucu]

Hi @ZhenboZhao77 and @July-1024,

We thank you for your interest in our work and apologize for the delayed response. For your convenience, we have just added the relevant detection files and some supplementary code where you can directly generate the Vanilla MoE and the state-of-the-art MoCaE results each with a single line of code. Please take a look at the Reproducing Rotated Object Detection Results part under the README and to the corresponding commit (9abce59f76055fd184e25efcf8ae59c27fce0dab) for more details. I hope that this simplifies the aforementioned process by a great margin and please ping us should you have any more questions or face any troubles.

[kemaloksuz]

@ZhenboZhao77 - Please refer here to reproduce our results. I am closing the issue now. Feel free to let us know and reopen this issue if you have any questions or difficulties.