How to deal with rotated bounding boxes in custom dataset?

NVIDIA / DIGITS

Deep Learning GPU Training System

https://developer.nvidia.com/digits

BSD 3-Clause "New" or "Revised" License

4.12k stars 1.38k forks source link

How to deal with rotated bounding boxes in custom dataset? #1487

Open hadign20 opened 7 years ago

hadign20 commented 7 years ago

In object detection, the required format for labels is the format of KITTI in which, one has to put the left, top, right, bottom pixel coordinates for each bounding box. I'm trying to perform a vehicle detection on aerial images, as in https://devblogs.nvidia.com/parallelforall/deep-learning-object-detection-digits/. But in my dataset, the label files contain the coordinates of 4 corners of each bounding box and the angle of each vehicle. How can I deal with objects with different angles, varying from 0 to 360 degrees? Is Digits suitable for dealing with rotation invariant object detection?

Thank you.

rodrigoberriel commented 7 years ago

AFAIK, based on how DetectNet works, it's not ready for that. In fact, most DNN-based object detection works just like DetectNet, so you might need to hack the code and perform some experiments, i.e. research. If you modify DetectNet to achieve this, please share your findings with us.

lukeyeager commented 7 years ago

Well said @rodrigoberriel. DetectNet is built to detect vertically (and horizontally?) aligned rectangles. You'll need to design a different network if you want to detect rotated rectangles, ellipses, general polygons, etc. DIGITS should be able to run any of those alternate networks you may come up with, with two caveats:

The data-loading plugin for detectnet is rectangle-based. To detect another type of shape, you'll either have to create the LMDBs yourself or make a new data-loading plugin to load your new data type.
The activation visualization plugin for detectnet is also rectangle-based. To visualize another type of output, you'd have to make a new visualization plugin.

hadign20 commented 7 years ago

Thank you for the helpful response. I just have some extra questions: 1- How to import LMDB dataset directly to the net in Digits? 2- Is Digits capable of training any custom models other than DetectNet? I know it doesn't support Faster RCNN. 3- What do you suggest I do to implement a rotation invariant object detection? Do you know any examples?

Thank you and I'm sorry for asking too many question..

happygao commented 6 years ago

@hadi-ghnd Hello, I am using the DIGITS to do the experiments on detect, and I meet the same questions with you, Do you have a method to these questions? I hope we could talk about on this question.