AssertionError when working with ImageDataGenerator

[EXJUSTICE]

I've been trying to implement IMGAUG to work with Keras's ImageDataGenerator (and have found the thread where this is discussed), but am running into an Assertion Error with no explanation during the fit command

history = model.fit_generator(train_generator, steps_per_epoch = steps_per_epoch, epochs=epochs, workers=4, validation_data=validation_generator, validation_steps=validation_steps)

This yields the error:

AssertionError                            Traceback (most recent call last)
<ipython-input-39-e7ff542a51eb> in <module>()
     10                               workers=4,
     11                               validation_data=validation_generator,
---> 12                               validation_steps=validation_steps)

18 frames
/usr/local/lib/python3.6/dist-packages/imgaug/augmenters/blend.py in blend_alpha(image_fg, image_bg, alpha, eps)
     94     """
     95     assert image_fg.shape == image_bg.shape
---> 96     assert image_fg.dtype.kind == image_bg.dtype.kind
     97     # TODO switch to gate_dtypes()
     98     assert image_fg.dtype.name not in ["float128"]

`

I think this has to do with my images not being in the numpy unit8 format I believe, but I'm not fully sure about this, as documentation says that at times it will work with floats too? Any advice is appreciated

[aleju]

That function tries to blend two images using a factor, i.e. something like factor * image_foreground + (1-factor) * image_background. Before doing that, it verifies that foreground and background image arrays have both the same numpy dtype and that's where it fails. If both inputs are float dtypes it should work perfectly fine, but for some reason one of them isn't a float.

Is that the whole stacktrace? It looks like there should be more. Which augmenters are you using? Which datatype exactly? I doubt that it has anything to do with the Keras data generator.

[EXJUSTICE]

Hi Alex

Thank you for the rapid response. and deep insight I see, I wonder why that is. I was working with the vanilla stanford cars-196 dataset. The only function prior to augmentation was a YOLO-based cropping function that crops out the background, returning images (of dtype 'numpy.ndarray' ) which I zipped up for later use during training.

The imgaug methods I had been using were Sharpen and AddToHueAndSaturation

I suspect you may be right, as the classifier works fine when using tensorflow.image augmentations and keras-based augmentations only.

[aleju]

AddToHueAndSaturation is the culprit here as that augmenter is based on ChangeColorspace, which again assumes inputs to be uint8 images and always converts its results to uint8 before blending them with the input. So float32 images (which you probably have) would be colorspace-converted, then converted to uint8, then blended with float32 input images, followed by the crash. In the latest version from master AddToHueAndSaturation actually rejects float-dtypes. Will have to add that to ChangeColorspace too. There is also an overview of which dtypes should work with which augmenter: https://imgaug.readthedocs.io/en/latest/source/dtype_support.html