image data augmentation

[kmsravindra]

@AlexeyAB , Went through the issue log and understood that random cropping, multi-scaling, color and lighting aspects of data augmentation has been already taken care in this repo. Wondering if there is a plan to cover other types as well enabling data aug on the fly ? like - random rotations, flips, perspective transforms etc.,? (Maybe at least rotations and flips).

[AlexeyAB]

• horizontal flips - aready implemented, and enabled by default. To disable use flip=0 in the [net]-section in cfg-file

• vertical flips - I do not know where it could help.

• rotations - is in the plans, but has not yet found the datasets in which it was required, except satellite imagery

• perspective transforms - since this will change the entire image, it will create disproportional distortions for near and far objects. I do not know the datasets for which this could help.

[kmsravindra]

@AlexeyAB, I thnk rotations, vertical flips and perspective transforms could be helpful in medical imaging. Given the possibillity of camera being titled at any angle + having limited access to training data would make a good case for such augmentation.

[xiaoyongzhu]

@kmsravindra on a different thread, can you point me to the multi-scaling part of the code, or where is the github issue discussing this? I am interested to learn more about the multi-scale data augmentation, but cannot find any docs/issues talking about it.

Thanks!

[AlexeyAB]

@xiaoyongzhu

• random=1 in cfg-file - resizes neural network size: https://github.com/AlexeyAB/darknet/blob/6390a5a2ab61a0bdf6f1a9a6b4a739c16b36e0d7/src/detector.c#L132-L151 or the same but by using OpenCV (3.5x times faster): https://github.com/AlexeyAB/darknet/blob/6390a5a2ab61a0bdf6f1a9a6b4a739c16b36e0d7/src/http_stream.cpp#L256

---

• jitter=0.1 - jitter=0.49 - resizes image size: https://github.com/AlexeyAB/darknet/blob/6390a5a2ab61a0bdf6f1a9a6b4a739c16b36e0d7/src/data.c#L743-L793

[xiaoyongzhu]

@AlexeyAB Very helpful! Thanks for the links!

[kmsravindra]

@AlexeyAB, I have implemented a standalone rotation augmentation in python. But this creates additional images in the training data thus occupying lot of storage. I wanted to incorporate this into the code that does the rotation augmentation on the fly during the training run time. Could you point me to some code in this rep where I can include my augmentation code to run it on the fly?

[AlexeyAB]

@kmsravindra

You should rotate both Images and Labels (bounded boxes).

1. If OpenCV is used - 3.5x accelerated data augmentation for training on High Performance GPU - TitanV, DGX-2,...:
   • https://github.com/AlexeyAB/darknet/blob/b847f39f60eb6715325f3707e78667a0611811dd/src/data.c#L723-L789
   • https://github.com/AlexeyAB/darknet/blob/b847f39f60eb6715325f3707e78667a0611811dd/src/http_stream.cpp#L272-L328

---

2. If OpenCV isn't used: https://github.com/AlexeyAB/darknet/blob/b847f39f60eb6715325f3707e78667a0611811dd/src/data.c#L791-L842

[NEELMCW]

@AlexeyAB is online rotation augmentation supported in current trunk ?

[deimsdeutsch]

@AlexeyAB Is random rotation merged in the code ?