CutMix and Mosaic Augmentation

albumentations-team / albumentations

Fast and flexible image augmentation library. Paper about the library: https://www.mdpi.com/2078-2489/11/2/125

https://albumentations.ai

MIT License

14.29k stars 1.65k forks source link

CutMix and Mosaic Augmentation #677

Open awsaf49 opened 4 years ago

awsaf49 commented 4 years ago

📚 Documentation

CutMix and Mosaic Augmentations are pretty good augmentation when it comes to achieve better score. It would be great if these two are included in albumentations.

ternaus commented 4 years ago

Could you please provide links to the description of Mosaic Augmentation?

awsaf49 commented 4 years ago

Could you please provide links to the description of Mosaic Augmentation?

I think it was even mentioned in YOLOv4 paper. Here are some references: https://www.kaggle.com/nvnnghia/awesome-augmentation https://github.com/klauspa/Yolov4-tensorflow

chamorajg commented 3 years ago

Is someone working on this issue ? Can I pick this up and create a PR for this request ?

Dipet commented 3 years ago

Yes, you can try to implement this.

senarvi commented 3 years ago

I guess what makes it difficult is that you would need to implement an augmentation that takes multiple images as input. Is that even possible in Albumentations?

mulcyber commented 3 years ago

The BasicTransform (and all other transforms) take only 1 image a input and don't manage iterables themselves. So I don't think it's possible without changing how albumentations or putting them as a different kind of object.

We would need to create an "BasicIterableTransform" than takes an iterable as input. But it wouldn't work with tools like torch.utils.data.Dataset without modification, need a new compose class, etc.

I agree it's a shame they're not included. It's super useful in detection problems, even though it requires quite a few modifications in albumentations and changes how the transforms would be used.

Any maintainer can tell us if it's the kind of features (namely Transforms taking iterables) that would be accepted/merged or it's just too different to how the library works now?

dav-ell commented 2 years ago

I've been working on a custom augment that performs mosaic by loading in 3 new images with whatever image is currently in the pipeline, but I got stuck at trying to return the new set of bounding boxes in the final moasic-ed image. Has anyone made progress on this?

dav-ell commented 2 years ago

I was able to create a custom version that fit my needs, but I couldn't get it to work with the albumentations pipeline. I ended up doing this (where self.mosaic :

# in my datamodule

self.mosaic = Mosaic(img_files, bbox_files, p=0.9) # requires all image and label files to create random mosaics

def transform(image, bboxes, class_labels, **params):
    """A wrapper for the albumentations pipeline"""
    image, bboxes, class_labels = self.mosaic(image, bboxes, class_labels)
    return self.transform(image=image, bboxes=bboxes, class_labels=class_labels, **params)

# create dataset, pass transform

That way my dataset could still call transform() with the usual albumentations API. However, it'd be nice to not have to use workarounds like this.

i-aki-y commented 2 years ago

I made a PR to add Mosaic augmentation with as few changes to the existing implementation as possible #1147. Although some remaining works are to be done, please check it out if you are interested in this feature. I welcome any comments and feedback.