keras-team / keras-preprocessing

Utilities for working with image data, text data, and sequence data.
Other
1.02k stars 444 forks source link

Missing scale parameter in apply_brightness_shift() call #327

Closed Mantas-Skackauskas closed 3 years ago

Mantas-Skackauskas commented 3 years ago

Problem description

I was using ImageDataGenerator with the parameter 'brightness_range'. This lead to unexpected behaviour where using the brightness range in interval [1,1] altered the image even though nothing should have happened. Looking more into the code, I noticed that the brightness range function generates a value 'brightness' from the uniform distribution with interval specified in the 'brightness_range' parameter, hence, the only value from U(1,1) must be 1. However, when the generated value 'brightness' is passed into the function apply_brightness_shift at line 888 seen below

https://github.com/keras-team/keras-preprocessing/blob/58df11e1145b2088092252c4dba02168c6da2b13/keras_preprocessing/image/image_data_generator.py#L888

the scale parameter is ignored and leads to the function using the default parameter 'scale'=True seen below:

https://github.com/keras-team/keras-preprocessing/blob/58df11e1145b2088092252c4dba02168c6da2b13/keras_preprocessing/image/affine_transformations.py#L215

This leads to a scaled output in range from [0-255] which alters the original image value range image-wise, and therefore performs unintentional min-max scaling to the input. Considering this function should only perform brightness modification and not per image min-max scaling of the values, such unexpected behaviour should not happen.

How I found this issue

I found this issue while using TensorFlow (2.3.1) package with function ImageDataGenerator which still relies on calls to keras API and took some time to locate where the issue is coming from.

How to replicate the problem

from tensorflow.keras.preprocessing.image import ImageDataGenerator
import numpy as np

old_image = np.random.randint(low=20,high=50,size=(1,10,10,1),dtype = 'uint8')

idg = ImageDataGenerator(brightness_range=(1,1)).flow(old_image) # unexpected behaviour

new_image = next(idg)[0].astype('uint8')

# output in range [20,50]
print(f'Original value range {old_image.min()} to {old_image.max()}')
# alters image and min-max scales it to range [0,255]
print(f'Original value range {new_image.min()} to {new_image.max()}')

This leads to the altered image:

Original value range 20 to 49 Original value range 0 to 255

Solution

I propose to set the scale parameter in the line 888 to False which will perform the brightness_range without scaling and will return the expected outcome:

 x = apply_brightness_shift(x, transform_parameters['brightness'], False) 
Mantas-Skackauskas commented 3 years ago

Created a pull request at: https://github.com/keras-team/keras-preprocessing/pull/328