Mosaic data augmentation

[LucasSloan]

In YOLO v4, they used a data augmentation technique they call mosaic. With the mosaic augmentation, you mix together 4 images to produce a single image. In the YOLO v4 paper they suggest that this has 2 effects - first, it helps the network learn to recognize objects outside of the contexts found in your data set, and second, it helps improve the batchnorm statistics when using small minibatches, since each image averages over several image's statistics.

The darknet implementation is here.

[fsx950223]

I have add augmix to the repo, but something wrong when I try to mix bboxs.

[wuting3]

I fond a mistake when resize_and_crop_boxes you use
boxes = preprocessor.box_list_scale( boxlist, self._scaled_height, self._scaled_width).get() but i think you should change it to boxes = preprocessor.box_list_scale( boxlist, self._image_scale, self._image_scale).get()

in addition when you clip box, you use def clip_boxes(self, boxes): """Clip boxes to fit in an image.""" boxes = tf.where(tf.less(boxes, 0), tf.zeros_like(boxes), boxes) boxes = tf.where(tf.greater(boxes, self._output_size[0] - 1), (self._output_size[1] - 1) * tf.ones_like(boxes), boxes)

That only can be used for rectangle that width=height,you can change it to : def clip_boxes(self, boxes): """Clip boxes to fit in an image.""" y_min, x_min, y_max, x_max = tf.split(value=boxes, num_or_size_splits=4, axis=1) y_min_clipped = tf.maximum(tf.minimum(tf.squeeze(y_min), self._output_size[0]), 0) y_max_clipped = tf.maximum(tf.minimum(tf.squeeze(y_max), self._output_size[0]), 0) x_min_clipped = tf.maximum(tf.minimum(tf.squeeze(x_min), self._output_size[1]), 0) x_max_clipped = tf.maximum(tf.minimum(tf.squeeze(x_max), self._output_size[1]), 0) boxes = tf.stack([y_min_clipped,x_min_clipped,y_max_clipped,x_max_clipped],1) return boxes

[vidalmaxime]

What are the issues with the bboxes with augmix @fsx950223 ?

[fsx950223]

What are the issues with the bboxes with augmix @fsx950223 ?

It doesn't support mix bboxs. Contribution welcome.

[qtw1998]

In YOLO v4, they used a data augmentation technique they call mosaic. With the mosaic augmentation, you mix together 4 images to produce a single image. In the YOLO v4 paper they suggest that this has 2 effects - first, it helps the network learn to recognize objects outside of the contexts found in your data set, and second, it helps improve the batchnorm statistics when using small minibatches, since each image averages over several image's statistics.

The darknet implementation is here.

Have you wrote it? I realise mosaic in pytorch Here is my codes: https://github.com/qtw1998/augment-efficientdet-AnchorFree/blob/f7f327a4604a2505ef475f8e90f304037b14a43a/efficientdet/dataset.py#L167

But I could not write in tf, I know little about tf.

[zishanahmed08]

@fsx950223 If i understand right, augmix is still faulty and i should not use it?

[fsx950223]

@fsx950223 If i understand right, augmix is still faulty and i should not use it?

I'm not recommended.

[evanmartua34]

Hi I wonder if augmix still a problem and cannot be use?

[kartik4949]

@LucasSloan @evanmartua34 @zishanahmed08 i have completed the mosaic augmentation in core tf2 code without numpy or cv2 please view my latest PR #701 and any suggestions are welcome here im resizing the images rather than cropping them and currently supporting exact four images.