keras-team / keras-applications

Reference implementations of popular deep learning models.
Other
2k stars 914 forks source link

Inconsistent imagenet preprocessing function #128

Open lagejoao opened 4 years ago

lagejoao commented 4 years ago

When preprocessing an image using a tf.Tensor and a np.array I get different behaviours for the same representation.

import requests
from imageio import imread
from io import BytesIO

r = requests.get("https://images-na.ssl-images-amazon.com/images/I/71j7T9vTmZL._SY355_.png")
img = imread(BytesIO(r.content))  # img.dtype = uint8

import tensorflow as tf
tf.enable_eager_execution()

x_tf = tf.constant(img)

out_np = tf.keras.applications.resnet50.preprocess_input(img)
out_tf = tf.keras.applications.resnet50.preprocess_input(x_tf).numpy()

np.testing.assert_array_equal(out_np, out_tf)  # 100% mismatch
np.testing.assert_array_equal(out_np.astype("uint8"), out_tf) # 32.19% mismatch

When casting the numpy output to type uint8 we will get a closer representation to the symbolic output.


I see a possible issue on https://github.com/keras-team/keras-applications/blob/master/keras_applications/imagenet_utils.py#L143 where there's a cast on the wrong tensor. Shouldn't it be

x = backend.bias_add(
            backend.cast(x, backend.dtype(mean_tensor)),
            mean_tensor,
            data_format=data_format,
        )

instead of casting the mean_tensor as uint8 with the following

x = backend.bias_add(
            x,
            backend.cast(mean_tensor, backend.dtype(x)),
            data_format=data_format)
lagejoao commented 4 years ago

@tanzhenyu I've realised you changed some logic quite recently https://github.com/keras-team/keras-applications/commit/e52c477a9ccb86b4c7bef7569d59f0c0b2e01cb6. Any input here?

tanzhenyu commented 4 years ago

Yep I think it should be other way around, I will make a change

lagejoao commented 4 years ago

I'll do it, then. Ok for you?

tanzhenyu commented 4 years ago

Sure, thanks