Add `by_name=True` when loading imagenet weights

[jrdi]

I am trying to run a model that performs a simple preprocessing to the input before passing it to the encoder. I created a dummy version for illustration purpose.

import keras

img = keras.layers.Input(shape=(224, 224, 3), name='img')
x = keras.layers.Dense(3)(x)

model = keras.applications.vgg16.VGG16(
    input_tensor=x,
    weights='imagenet',
    include_top=False)

model.summary()

This code raises an exception due to the extra layer.

ValueError: You are trying to load a weight file containing 13 layers into a model with 14 layers.

We can easily fix the issue using by_name=True when we load imagenet weights, hence here is a PR changing the current behavior. Is there anything dangerous that I am missing?

[taehoonlee]

Thank you for the PR, @jrdi. Weight names in the definition and the pretrained file may not match. Generally, the argument by_name=False is safer than True.

[see--]

To avoid by_name=True, just construct a new model

import keras

img = keras.layers.Input(shape=(224, 224, 3), name='img')
x = keras.layers.Dense(3)(img)

model = keras.models.Model(img, keras.applications.vgg16.VGG16(
    weights='imagenet',
    include_top=False)(x))

print(model.summary())

[jrdi]

Thank you for your reply, @taehoonlee. I understand your point, but then, how will you approach the situation I'm trying to solve? I also tried to create the encoder and then modify the resulting model adding the extra layer but I feel the resultant code (and the resultant nested model) are not clean enough.

Edit: Thank you, @see--! Do you think it's a good solution? Having a nested model adds an extra layer of complexity to the model (many libraries, especially exporters, doesn't have support) for a simple operation such as adding a Dense layer before the encoder input. What do you think?

[see--]

Yes, it's a good solution. Do you have an example for missing support? I agree that the model.summary() output doesn't look that nice anymore, but it should work fine and is tested e.g. https://github.com/keras-team/keras/blob/master/tests/test_model_saving.py#L584.

[jrdi]

@see-- when I said missing support, I was talking about external libraries support, here you have an example: https://github.com/apple/coremltools/issues/112

[taehoonlee]

@jrdi, You can use model as a functional API.

import keras

img = keras.layers.Input(shape=(224, 224, 3), name='img')
x = keras.layers.Dense(3)(img)

base_model = keras.applications.vgg16.VGG16(
    weights='imagenet',
    include_top=False)

new_output = base_model(x)
model = keras.models.Model(inputs=img, outputs=new_output)

Test codes:

model.layers[1].set_weights([np.eye(3, dtype=np.float32), np.zeros(3, dtype=np.float32)])
data = np.random.random((1, 224, 224, 3)).astype(np.float32)
y1 = base_model.predict(data)
y2 = model.predict(data)
np.testing.assert_allclose(y1, y2)

[jrdi]

@taehoonlee @see-- thank you. I've been playing around with the solution using nested model and seems to work well, I will move forward using this implementation, so feel free to close the PR if you want. By the way, just to double check, I'm doing this to introduce the image preprocessing as a layer of my model, this way I can use it in production keeping this logic inside the model + the preprocess is always done by the GPU (if available). Do you think is a good idea? I'm wondering why keras doesn't have a layer for doing so.

The implementation is as simple as:

layer = Dense(channels, trainable=False, name='image_preprocessing')
x = layer(img)

layer.set_weights([
    np.diag([scaling, scaling, scaling]),
    np.array([r_bias, g_bias, b_bias])
])

[see--]

You could use a Lambda layer

def image_preprocessing(x):
  r_bias = 1.0
  g_bias = 2.0
  b_bias = 3.0
  scaling = 4.0
  return (x - [r_bias, g_bias, b_bias]) / scaling

x = keras.layers.Lambda(image_preprocessing)(img)

[jrdi]

Of course, I can use a lambda but in @fchollet words:

The problem with using a Lambda, is that they are only serializable via bytecode, which will severely limit the portability of the model (for instance, it could not be exported to TF.js or to CoreML).

They are not easy to export/import to other frameworks without having to implement a custom, hence there is no advantage of using a lambda layer over a function to preprocess the input image (in terms of keeping the preprocessing logic inside the model).

[see--]

Interesting. Then your solution seems good. FYI you can use the Lambda layer with tflite on Android.

[taehoonlee]

@jrdi, A combination of keras.layers.Subtract() and keras.layers.Multiply() is another option.