3D CAM can not work （visualize_cam（））

[WAMAWAMA]

error log ：

Traceback (most recent call last):
  File "E:\pycharm\jtbrain\PyCharm 2018.1.4\helpers\pydev\pydev_run_in_console.py", line 52, in run_file
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "E:\pycharm\jtbrain\PyCharm 2018.1.4\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "G:/@diploma_project/data&code/@code_wang/dl_py_project/test1.py", line 53, in <module>
    grads = visualize_cam(model, 22, filter_indices=0, seed_input=data_input_c[0,:,:,:,:])
  File "E:\Anaconda\ANACONDA\lib\site-packages\vis\visualization\saliency.py", line 239, in visualize_cam
    return visualize_cam_with_losses(model.input, losses, seed_input, penultimate_layer, grad_modifier)
  File "E:\Anaconda\ANACONDA\lib\site-packages\vis\visualization\saliency.py", line 186, in visualize_cam_with_losses
    heatmap = imresize(heatmap, input_dims, interp='bicubic', mode='F')
  File "E:\Anaconda\ANACONDA\lib\site-packages\numpy\lib\utils.py", line 101, in newfunc
    return func(*args, **kwds)
  File "E:\Anaconda\ANACONDA\lib\site-packages\scipy\misc\pilutil.py", line 555, in imresize
    im = toimage(arr, mode=mode)
  File "E:\Anaconda\ANACONDA\lib\site-packages\numpy\lib\utils.py", line 101, in newfunc
    return func(*args, **kwds)
  File "E:\Anaconda\ANACONDA\lib\site-packages\scipy\misc\pilutil.py", line 327, in toimage
    raise ValueError("'arr' does not have a suitable array shape for "
ValueError: 'arr' does not have a suitable array shape for any mode.

whole test code :

import numpy as np
from vis.visualization import visualize_cam
from keras.models import *
from keras.layers import Input, Flatten, Dense, Conv3D, MaxPooling3D
from keras.optimizers import SGD
def vgg16_w_3d(classes):
    inputs = Input(shape=(224, 224, 15, 1), name='input')
    # Block 1
    x = Conv3D(64, (3, 3, 3), activation='relu', padding='same', name='block1_conv1')(inputs)
    x = Conv3D(64, (3, 3, 3), activation='relu', padding='same', name='block1_conv2')(x)
    x = MaxPooling3D((2, 2, 2), strides=(2, 2, 2), name='block1_pool')(x)
    # Block 2
    x = Conv3D(128, (3, 3, 3), activation='relu', padding='same', name='block2_conv1')(x)
    x = Conv3D(128, (3, 3, 3), activation='relu', padding='same', name='block2_conv2')(x)
    x = MaxPooling3D((2, 2, 2), strides=(2, 2, 1), name='block2_pool')(x)
    # Block 3
    x = Conv3D(256, (3, 3, 3), activation='relu', padding='same', name='block3_conv1')(x)
    x = Conv3D(256, (3, 3, 3), activation='relu', padding='same', name='block3_conv2')(x)
    x = Conv3D(256, (3, 3, 3), activation='relu', padding='same', name='block3_conv3')(x)
    x = MaxPooling3D((2, 2, 2), strides=(2, 2, 1), name='block3_pool')(x)
    # Block 4
    x = Conv3D(512, (3, 3, 3), activation='relu', padding='same', name='block4_conv1')(x)
    x = Conv3D(512, (3, 3, 3), activation='relu', padding='same', name='block4_conv2')(x)
    x = Conv3D(512, (3, 3, 3), activation='relu', padding='same', name='block4_conv3')(x)
    x = MaxPooling3D((2, 2, 2), strides=(2, 2, 1), name='block4_pool')(x)
    # Block 5
    x = Conv3D(512, (3, 3, 3), activation='relu', padding='same', name='block5_conv1')(x)
    x = Conv3D(512, (3, 3, 3), activation='relu', padding='same', name='block5_conv2')(x)
    x = Conv3D(512, (3, 3, 3), activation='relu', padding='same', name='block5_conv3')(x)
    x = MaxPooling3D((2, 2, 2), strides=(2, 2, 2), name='block5_pool')(x)
    # dense
    x = Flatten(name='flatten')(x)
    x = Dense(4096, activation='relu', name='fc1')(x)
    x = Dense(4096, activation='relu', name='fc2')(x)
    x = Dense(classes, activation='softmax', name='predictions')(x)
    model = Model(inputs=inputs, outputs=x, name='vgg16')
    model.compile(optimizer=SGD(lr=1e-6), loss='categorical_crossentropy', metrics=['acc'])
    return model

model = vgg16_w_3d(3)
data_input_c = np.zeros([1,224,224,15,1], dtype=np.float32)  # net input container
grads = visualize_cam(model, 22, filter_indices=0, seed_input=data_input_c[0,:,:,:,:])

version:

keras  2.1.0
keras-vis  0.4.1
tensorflow  1.4.0
numpy  1.12.0
scipy  1.2.1

[WAMAWAMA]

OK, I find the reason, though it's so weird. The package which I downloaded by pip install Keras-vis in Windows10 is different from the one on GitHub.

suchlike the following:

the vis\visualization\saliency.py download by pip install Keras-vis in Windows10

    # ReLU thresholding to exclude pattern mismatch information (negative gradients).
    heatmap = np.maximum(heatmap, 0)

    # The penultimate feature map size is definitely smaller than input image.
    input_dims = utils.get_img_shape(input_tensor)[2:]
    heatmap = imresize(heatmap, input_dims, interp='bicubic', mode='F')

    # Normalize and create heatmap.
    heatmap = utils.normalize(heatmap)
    return np.uint8(cm.jet(heatmap)[..., :3] * 255)

the vis\visualization\saliency.py one on GitHub.

    # ReLU thresholding to exclude pattern mismatch information (negative gradients).
    heatmap = np.maximum(heatmap, 0)

    # The penultimate feature map size is definitely smaller than input image.
    input_dims = utils.get_img_shape(input_tensor)[2:]

    # Figure out the zoom factor.
    zoom_factor = [i / (j * 1.0) for i, j in iter(zip(input_dims, output_dims))]
    heatmap = zoom(heatmap, zoom_factor)
    return utils.normalize(heatmap)

its easy to find the difference.

[keisen]

Hi, @WAMAWAMA . Thank you for reporting the problem and solution!

We'll close this issue because you've already done and there are some issues regarding the same problem (Such as #198 ). But please feel free to reopen this issue whenever you need.

Thanks!