'application_resnet()' example fails with TensorFlow keras implementation

[kevinushey]

Given this code:

# Sys.setenv(KERAS_IMPLEMENTATION = "keras")
library(keras)

# instantiate the model
model <- application_resnet50(weights = 'imagenet')

# load the image
img_path <- "elephant.jpg"
img <- image_load(img_path, target_size = c(224, 224))
x <- image_to_array(img)

# ensure we have a 4d tensor with single element in the batch dimension,
# the preprocess the input for prediction using resnet50
dim(x) <- c(1, dim(x))
x <- imagenet_preprocess_input(x)

# make predictions then decode and print them
preds <- model %>% predict(x)
imagenet_decode_predictions(preds, top = 3)[[1]]

TensorFlow Keras

> imagenet_decode_predictions(preds, top = 3)[[1]]
  class_name           class_description score
1  n02098286 West_Highland_white_terrier     1
2  n15075141               toilet_tissue     0
3  n02319095                  sea_urchin     0

Stand-alone Keras

> imagenet_decode_predictions(preds, top = 3)[[1]]
  class_name class_description     score
1  n02504458  African_elephant 0.4545720
2  n01871265            tusker 0.3656018
3  n02504013   Indian_elephant 0.1791181

Any idea what might be going on here?

[jjallaire]

I'm seeing the same behavior when executing from Python:

# use keras
from keras.applications.resnet50 import ResNet50
from keras.preprocessing import image
from keras.applications.resnet50 import preprocess_input, decode_predictions

# use tensorflow.contrib.keras
# from tensorflow.contrib.keras.python.keras.applications.resnet50 import ResNet50
# from tensorflow.contrib.keras.python.keras.preprocessing import image
# from tensorflow.contrib.keras.python.keras.applications.resnet50 import preprocess_input, decode_predictions

import numpy as np

model = ResNet50(weights='imagenet')

img_path = 'elephant.jpg'
img = image.load_img(img_path, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

preds = model.predict(x)

print('Predicted:', decode_predictions(preds, top=3)[0])

Stand-alone Keras

('Predicted:', [(u'n02504013', u'Indian_elephant', 0.91937912), 
                (u'n01871265', u'tusker', 0.070962951), 
                (u'n02504458', u'African_elephant', 0.0095201703)])

TensorFlow Keras

('Predicted:', [(u'n02098286', u'West_Highland_white_terrier', 1.0), 
                (u'n15075141', u'toilet_tissue', 0.0), 
                (u'n02319095', u'sea_urchin', 0.0)])

[jjallaire]

Reported to tensorflow here: https://github.com/tensorflow/tensorflow/issues/11868

[jjallaire]

I've resolved this for the Keras R package by switching the default implementation to "keras" rather than "tensorflow" here: https://github.com/rstudio/keras/pull/88