aleju / imgaug

Image augmentation for machine learning experiments.
http://imgaug.readthedocs.io
MIT License
14.36k stars 2.43k forks source link

Map (Mask) segmentation: fill newly created pixels with mode != "constant" [HAS WORKAROUND] #746

Open boriswinner opened 3 years ago

boriswinner commented 3 years ago

This has been already discussed here: https://github.com/aleju/imgaug/issues/499 https://github.com/aleju/imgaug/issues/710

The situation is: I want to augment a satellite image and its corresponding mask. For the image, I want to use "Warp" mode. This image is zoomed out and the new pixels are filled using "warp" mode: input_train2

However, by default, the mask will NOT be filled the same way. IMO, this leads to the incorrectness of the mask (i.e. the newly created pixels of the image have pieces from "road" and "background" classes, but on the mask it's all filled with zeros). The masks looks like this: prediction_2_real_class0

There is a workaround. When defining an instance of Affine, you need to set the _mode_segmentation_maps variable:

        affineTransformations = iaa.Affine(
                    scale={"x": (0.9, 1.1), "y": (0.9, 1.1)},
                    rotate=(-10, 10),
                    mode='wrap')
        affineTransformations._mode_segmentation_maps = 'wrap'

And then you can use it like this:

        seq = iaa.Sequential(
            [
                iaa.Fliplr(0.5),  # horizontally flip 50% of all images
                iaa.Flipud(0.2),  # vertically flip 20% of all images
                affineTransformations,
            ],
            random_order=True
        )
        images_aug_i, segmaps_aug_i = seq(images=images, segmentation_maps=segmaps)

This workaround works, but it would be better to have a clean solution in the library. The corresponding mask now looks like this: prediction_2_real_class0

edumotya commented 2 years ago

https://github.com/aleju/imgaug/issues/788