backward pass of rotated IoU to enable (G)IoU Loss

[tobiasfshr]

🚀 Feature

It would be great if the backward pass of the rotated IoU CUDA module would be implemented, since this way one could train a detector to directly optimize it. Moreover, recent papers introduced the generalized IoU metric, which better captures the quality of detections w.r.t. the ground truth boxes. Since the extension of (rotated) IoU to GIoU is relatively simple, it would be good to implement this too when implementing the backward pass.

Motivation & Examples

There a multiple papers that describe the advantage of detectors that are trained to directly optimize the (G)IoU in the box regression branch: https://arxiv.org/pdf/1902.09630.pdf, CVPR'19 Algorithm 1 in this paper best describes the GIoU metric Algorithm 2 describes how to use IoU / GIoU as loss for 2D object detection The experiments section validates the method, finding relative improvements of roughly 2-7% on COCO for multiple architectures (YOLOv3, FasterRCNN, MaskRCNN). Yolov4 uses this loss as well: https://arxiv.org/abs/2004.10934

https://arxiv.org/pdf/1908.03851v1.pdf, 3DV'19 Extends the (G)IoU loss to 3D object detection, would be easy to implement if rotated 2D (G)IoU would be available including backward pass. Shows similar performance gains. Section 4.2 describes (G)IoU as Loss.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6806216/, Sensors 2019 Describes the same idea as the paper above, including Algorithm 1 describing 3D GIoU loss. Figure 5 gives intuitive illustration of 3D GIoU.

Note

Since there a multiple papers that report similar performance gains independently from each other and on two different tasks (i.e. 2D and 3D object detection), I believe this feature could be of great use in detectron2.

Best, Tobias

[filaPro]

Agree, that this feature would be really useful.

[lanbing510]

Agree

[jonmorton]

GIoU loss support has been added for rpn and rcnn box head losses. see #1085 @ppwwyyxx maybe these two issues should be merged?

[ppwwyyxx]

Looks like this issue is more about making rotated box IOU differentiable - that's much harder than just adding a new type of losses

[yzheng97]

Looks like this issue is more about making rotated box IOU differentiable - that's much harder than just adding a new type of losses

It is addressed in this 3DV 2019 paper: https://arxiv.org/abs/1908.03851

But I could not find any analytical implementation of it. Quite intrival.

[maudzung]

Hi everyone,

I have implemented the IoU and GIoU calculations for 2 rotated boxes. The computations are done on GPU. Although I carefully checked it and compared the results with shapely library, please check it out and give me comments. Thank you so much!

For IoU: here For GIoU: here