albumentations-team / albumentations

Fast and flexible image augmentation library. Paper about the library: https://www.mdpi.com/2078-2489/11/2/125
https://albumentations.ai
MIT License
14.23k stars 1.65k forks source link

transformation is not applied to the mask equally #1346

Open YJonmo opened 1 year ago

YJonmo commented 1 year ago

🐛 Bug

After applying the transformations (most likely due to random_crop/blur/Downscale), the mask drifts. This drift can be seen in the image_after, particularly around the shoes and head.

              Image_before                                    image_after
image

To Reproduce

Steps to reproduce the behavior:

  1. aug_transforms.transforms: [Normalize(always_app...lue=255.0), RandomResizedCrop(al...olation=1), Downscale(always_app..._max=0.35), HorizontalFlip(alway...se, p=0.5), Blur(always_apply=Fa...it=(7, 7)), Blur(always_apply=Fa...it=(3, 3)), RandomBrightnessCont..._max=True), ChannelShuffle(alway...e, p=0.01)] special variables function variables 0:Normalize(always_apply=False, p=0.0, mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225), max_pixel_value=255.0) 1:RandomResizedCrop(always_apply=False, p=0.0, height=256, width=256, scale=(0.65, 1.2), ratio=(1.0, 1.0), interpolation=1) 2:Downscale(always_apply=False, p=1, scale_min=0.2, scale_max=0.35) 3:HorizontalFlip(always_apply=False, p=0.5) 4:Blur(always_apply=False, p=1, blur_limit=(7, 7)) 5:Blur(always_apply=False, p=0.75, blur_limit=(3, 3)) 6:RandomBrightnessContrast(always_apply=False, p=0.5, brightness_limit=(-0.15, 0.15), contrast_limit=(-0.1, 0.1), brightness_by_max=True) 7:ChannelShuffle(always_apply=False, p=0.01) len():8

to create the masked image: image_before = cv2.bitwise_and(fg_image, fg_image, mask=fg_mask) transformed = aug_transforms(image=fg_image, mask=fg_mask) fg_image = transformed["image"] fg_mask = transformed["mask"] image_after = cv2.bitwise_and(fg_image, fg_image, mask=fg_mask)

Expected behavior

Environment

YJonmo commented 1 year ago

Updates: it appears the issue of this shift is due to the default interpolation method for the alb.Downscale function. If interpolation=cv2.INTER_CUBIC or interpolation=cv2.INTER_LINEAR then the transformed image and mask overlap correctly.

And by the way, how to find out which of the transformations are applied using an instance created by my_instance = alb.Compose(transform_list), if all of the p in the transform_list are 0.5?

Dipet commented 1 year ago

And by the way, how to find out which of the transformations are applied using an instance created by my_instance = alb.Compose(transform_list), if all of the p in the transform_list are 0.5?

For this purpose try to use ReplayCompose