Trouble following example using custom model, using saliency()

[ianbgroves]

Hello,

I am following the Vanilla Saliency tutorial, trying to use my own pretrained model which is a categorical classification problem with 3 classes, on grayscale images of 200x200. To visualise the saliency map on an example input. I'm using Colab for this, you can see the notebook here. The saved model folder is here. And the sample image is here.

The issue is: When I run the following

from tf_keras_vis.utils.scores import CategoricalScore
score = CategoricalScore([0, 1, 2])
def score_function(output):
    # The `output` variable refers to the output of the model,
    # so,return the three values for the 1st, the 2nd and the 3rd of categories respectively.
   return (output[None, 0], output[None, 1], output[None, 2])

from tensorflow.keras import backend as K
from tf_keras_vis.saliency import Saliency
# from tf_keras_vis.utils import normalize

# Create Saliency object.
saliency = Saliency(model,
                    model_modifier=replace2linear,
                    clone=True)

# Generate saliency map
saliency_map = saliency(score_function, im)
#saliency_map = saliency(score, im)

I get:

---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-64-a44d39894abd> in <module>()
      9 
     10 # Generate saliency map
---> 11 saliency_map = saliency(score_function, im)
     12 #saliency_map = saliency(score, im)
     13 

12 frames
/usr/local/lib/python3.7/dist-packages/six.py in raise_from(value, from_value)

InvalidArgumentError: slice index 1 of dimension 0 out of bounds. [Op:StridedSlice] name: strided_slice/

I get the same error if I use CategoricalScore([0,1,2]) and pass saliency the output of this. I'm sure that I'm misunderstanding the instructions here. But I'm afraid I've reached the limit of my understanding.

A pre-emptive thank you for any help you can give.

If it's useful the model summary is:

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_4 (Conv2D)            (None, 200, 200, 16)      160       
_________________________________________________________________
dropout_7 (Dropout)          (None, 200, 200, 16)      0         
_________________________________________________________________
max_pooling2d_6 (MaxPooling2 (None, 100, 100, 16)      0         
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 100, 100, 32)      4640      
_________________________________________________________________
dropout_8 (Dropout)          (None, 100, 100, 32)      0         
_________________________________________________________________
max_pooling2d_7 (MaxPooling2 (None, 50, 50, 32)        0         
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 50, 50, 64)        18496     
_________________________________________________________________
dropout_9 (Dropout)          (None, 50, 50, 64)        0         
_________________________________________________________________
max_pooling2d_8 (MaxPooling2 (None, 25, 25, 64)        0         
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 25, 25, 128)       73856     
_________________________________________________________________
dropout_10 (Dropout)         (None, 25, 25, 128)       0         
_________________________________________________________________
max_pooling2d_9 (MaxPooling2 (None, 12, 12, 128)       0         
_________________________________________________________________
conv2d_8 (Conv2D)            (None, 12, 12, 256)       295168    
_________________________________________________________________
dropout_11 (Dropout)         (None, 12, 12, 256)       0         
_________________________________________________________________
max_pooling2d_10 (MaxPooling (None, 6, 6, 256)         0         
_________________________________________________________________
conv-final (Conv2D)          (None, 6, 6, 512)         1180160   
_________________________________________________________________
dropout_12 (Dropout)         (None, 6, 6, 512)         0         
_________________________________________________________________
max_pooling2d_11 (MaxPooling (None, 3, 3, 512)         0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 4608)              0         
_________________________________________________________________
dense_2 (Dense)              (None, 512)               2359808   
_________________________________________________________________
dropout_13 (Dropout)         (None, 512)               0         
_________________________________________________________________
dense_3 (Dense)              (None, 3)                 1539      
=================================================================
Total params: 3,933,827
Trainable params: 3,933,827
Non-trainable params: 0

[ianbgroves]

Also, I'm deviating from the example by using

import PIL.Image
from matplotlib import pylab as P
def LoadImage(file_path):
  im = PIL.Image.open(file_path).convert('L')

  im = np.asarray(im)
  return im

def ShowImage(im, title='', ax=None):
  if ax is None:
    P.figure()
  P.axis('off')
  P.imshow(im)
  P.title(title)

# Load the image
import cv2
im_orig = LoadImage('/content/drive/MyDrive/9. ML project/August_ML_tests/Data/labeled_data/10_2/10.2_022.jpg')
im = cv2.resize(im_orig,(200,200))
im = np.reshape(im,(-1, 200,200, 1))

# Show the image
ShowImage(im_orig)

predictions = model.predict(im)
prediction_class = np.argmax(predictions[0])

print("Prediction class: " + str(prediction_class)) 

To load and preprocess my image.

[ianbgroves]

Hi all, in case anyone else has this issue. The follow addition solved my issue:

X = np.reshape(X,(-1, 200,200, 1))

from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.applications.vgg16 import preprocess_input

# Image titles
image_titles = ['10.2', '10.2', '10.2']

# Load images and Convert them to a Numpy array
img1 = load_img('/content/drive/MyDrive/9. ML project/August_ML_tests/Data/labeled_data/10_2/10.2_009.jpg', grayscale=True, target_size = (200,200))
img2 = load_img('/content/drive/MyDrive/9. ML project/August_ML_tests/Data/labeled_data/10_2/10.2_009.jpg', grayscale=True, target_size = (200,200))
img3 = load_img('/content/drive/MyDrive/9. ML project/August_ML_tests/Data/labeled_data/10_2/10.2_009.jpg', grayscale=True, target_size= (200,200))
images = np.asarray([np.array(img1), np.array(img2), np.array(img3)])

# Preparing input data for VGG16
X = preprocess_input(images)
X = np.reshape(X,(-1, 200,200, 1))

# Rendering
f, ax = plt.subplots(nrows=1, ncols=3, figsize=(12, 4))
for i, title in enumerate(image_titles):
    ax[i].set_title(title, fontsize=16)
    ax[i].imshow(images[i])
    ax[i].axis('off')
plt.tight_layout()
plt.show()