Some thoughts on data augmentation

Zengyf-CVer commented 2 years ago

Search before asking

[X] I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

@glenn-jocher For the data augmentation part in hyp.scratch.yaml, I have two questions:

For translate the value of this parameter is 0.1, how to translate? How much is the translation along the x-axis or the y-axis? https://github.com/ultralytics/yolov5/blob/69032519bc575ef6c2033ab0f7d9bc1f9651b251/data/hyps/hyp.scratch.yaml#L26
Why does YOLOv5 have no data augmentationof Gaussian noise? Is it because Gaussian noise will affect the image quality, is the image distortion, and then reduce the overall mAP?

Additional

No response

glenn-jocher commented 2 years ago

@Zengyf-CVer hyp files contain most of the basic augmentations needed, and for extra augmentations like Gaussian noise you can enable these with Albumentations in augmentations.py:

https://github.com/ultralytics/yolov5/blob/def7a0fd19c1629903c3b073b4df265407719a07/utils/augmentations.py#L16-L17

See https://github.com/ultralytics/yolov5/pull/3882 for details.

I'm super excited to announce our new YOLOv5 🚀 + Albumentations integration!! Now you can train the world's best Vision AI models even better with custom Albumentations automatically applied 😃!

PR https://github.com/ultralytics/yolov5/pull/3882 implements this integration, which will automatically apply Albumentations transforms during YOLOv5 training if albumentations>=1.0.3 is installed in your environment.

Get Started

To use albumentations simply pip install -U albumentations and then update the augmentation pipeline as you see fit in the new Albumentations class in yolov5/utils/augmentations.py. Note these Albumentations operations run in addition to the YOLOv5 hyperparameter augmentations, i.e. defined in hyp.scratch.yaml.

Here's an example that applies Blur, MedianBlur and ToGray albumentations in addition to the YOLOv5 hyperparameter augmentations normally applied to your training mosaics :)
class Albumentations:
    # YOLOv5 Albumentations class (optional, used if package is installed)
    def __init__(self):
        self.transform = None
        try:
            import albumentations as A
            check_version(A.__version__, '1.0.3')  # version requirement

            self.transform = A.Compose([
                A.Blur(blur_limit=50, p=0.1),
                A.MedianBlur(blur_limit=51, p=0.1),
                A.ToGray(p=0.3)],
                bbox_params=A.BboxParams(format='yolo', label_fields=['class_labels']))

            logging.info(colorstr('albumentations: ') + ', '.join(f'{x}' for x in self.transform.transforms))
        except ImportError:  # package not installed, skip
            pass
        except Exception as e:
            logging.info(colorstr('albumentations: ') + f'{e}')

    def __call__(self, im, labels, p=1.0):
        if self.transform and random.random() < p:
            new = self.transform(image=im, bboxes=labels[:, 1:], class_labels=labels[:, 0])  # transformed
            im, labels = new['image'], np.array([[c, *b] for c, b in zip(new['class_labels'], new['bboxes'])])
        return im, labels
Example Result

Example train_batch0.jpg on COCO128 dataset with Blur, MedianBlur and ToGray. See the YOLOv5 Notebooks to reproduce:

Zengyf-CVer commented 2 years ago

@glenn-jocher After I installed albumations, how do I use albumations in yolov5? What command is entered in train.py? Or modify what configuration file? thanks.

glenn-jocher commented 2 years ago

@Zengyf-CVer you just pip install albumentations, and then insert any transforms you want in the Albumentations class here. YOLOv5 will automatically use albumentations when installed, no special commands are required.

We have a few albumentations enabled by default with low probability of 0.01: https://github.com/ultralytics/yolov5/blob/def7a0fd19c1629903c3b073b4df265407719a07/utils/augmentations.py#L24-L31

Zengyf-CVer commented 2 years ago

@glenn-jocher Have you ever encountered such a problem: After executing this command pip install albumentations, an opencv-python-headless will be installed, but this opencv-python-headless will conflict with the previous opencv-python, and then you can only uninstall the previousopencv-python, reinstall opencv-python, so that it can be used normally.

glenn-jocher commented 2 years ago

@Zengyf-CVer I haven't seen this problem with albumentations but in some other cases I have seen the various cv2 packages cause issues. If this is a problem in your environment then the best way to install albumentations would be together with all of the other pip package as the same time to allow the pip resolver to sort it all out:

pip install -r requirements.txt albumentations

or to upgrade everything:

pip install -U -r requirements.txt albumentations

Zengyf-CVer commented 2 years ago

@glenn-jocher Thank you very much.

Zengyf-CVer commented 2 years ago

@glenn-jocher What does this 0.01 mean? How does it affect the pictures in a batch? thanks. https://github.com/ultralytics/yolov5/blob/def7a0fd19c1629903c3b073b4df265407719a07/utils/augmentations.py#L24-L31

glenn-jocher commented 2 years ago

@Zengyf-CVer see https://albumentations.ai/docs/api_reference/augmentations/transforms/

Zengyf-CVer commented 2 years ago

@glenn-jocher Thank you very much.

ultralytics / yolov5