fill_mode on zoom and rotation problem

[nemanjaq]

I'm trying to augment both images and masks. Images are working propely but masks fail.

Example:

It happens when mask is zoomed out and rotated. That black and white should be blue, like background.

I've tried to change fill_mode, but it doesn't work for constant and nearest. Wrap works, but it creates red areas where it shouldn't.

Code:

def augGenerator():

    gen = ImageDataGenerator(
            rotation_range=20,
            shear_range=0.2,
            zoom_range=0.2,
            horizontal_flip=True,
            )

    return gen   

def augmentImage(img, mask, img_size, aug_count):

    aug_images = [img]
    aug_masks = [mask]

    img = img.reshape(-1, img_size, img_size, 3)   
    mask = mask.reshape(-1, img_size, img_size, 3)   

    gen_img = augGenerator()
    gen_mask = augGenerator()

    seed = 1

    gen_img.fit(img, augment=True, seed=seed)
    gen_mask.fit(mask, augment=True, seed=seed)

    img_aug_iter = gen_img.flow(img,seed=seed)
    mask_aug_iter = gen_mask.flow(mask,seed=seed)

    aug_images += [next(img_aug_iter)[0] for i in range(aug_count)]
    aug_masks += [next(mask_aug_iter)[0] for i in range(aug_count)]

    return aug_images, aug_masks 

[LeanderK]

we also experience the same bug! We are training a generative model, so this is VERY important for us!

[rragundez]

@nemanjaq @LeanderK It works for me but perhaps I'm missing something: This is a minimal example you can execute in a notebook cell

import os
import matplotlib.pyplot as plt
import numpy as np
import random
import matplotlib.pyplot as plt

%matplotlib inline

from keras_preprocessing.image import ImageDataGenerator

def augGenerator():
    gen = ImageDataGenerator(
            rotation_range=60,
            shear_range=0.2,
            zoom_range=[1.5, 2],
            horizontal_flip=True,
            )
    return gen   

def augmentImage(img, mask, img_size, aug_count):
    aug_images = [img]
    aug_masks = [mask]
    img = img.reshape(-1, img_size, img_size, 3)   
    mask = mask.reshape(-1, img_size, img_size, 3)   
    gen_img = augGenerator()
    gen_mask = augGenerator()
    seed = 1
    gen_img.fit(img, augment=True, seed=seed)
    gen_mask.fit(mask, augment=True, seed=seed)
    img_aug_iter = gen_img.flow(img,seed=seed)
    mask_aug_iter = gen_mask.flow(mask,seed=seed)
    aug_images += [next(img_aug_iter)[0] for i in range(aug_count)]
    aug_masks += [next(mask_aug_iter)[0] for i in range(aug_count)]
    return aug_images, aug_masks

image = np.stack((.3 * np.ones((200, 200)), .1 * np.ones((200, 200)), .5 * np.ones((200, 200))), axis=2)
image[30:170, 60:140] = .4
mask = np.zeros((200, 200, 3), dtype='float')
mask[50:150, 50:150] = 1.

aug_images, aug_masks = augmentImage(image, mask, 200, 1)
plt.imshow(np.hstack((aug_images)))
plt.show()
plt.imshow(np.hstack(aug_masks))

It works as expected with rotation and zoom out. Can you minimal modify that example to show the bug? I wan to see if I can help you out, if it's still an issue. Thanks