Attention maps - ValueError: operands could not be broadcast together

[walid0925]

Hi,

I am trying to use visualize_cam with a model that has been trained as channels_first, and I have been receiving the following error:

    266     # Create jet heatmap.
    267     heatmap_colored = np.uint8(cm.jet(heatmap)[..., :3] * 255)
--> 268     heatmap_colored = np.uint8(seed_img * alpha + heatmap_colored * (1. - alpha))
    269     return heatmap_colored

ValueError: operands could not be broadcast together with shapes (1,3,224,224) (224,224,3)

My preprocessing is as follows:

from keras import backend as K
import numpy as np
from scipy.misc import imresize, imread
from vis.utils import utils
from vis.visualization import visualize_saliency
from vis.visualization import visualize_cam

import my_model

# get Keras model
model = my_model.get_model()

K.set_image_data_format('channels_first')

heatmaps = []
for path in image_paths:
    x = imread(path)
    x = x.astype('float32')
    x = imresize(x, (224, 224, 3))

    # change BGR -> RGB
    x[:,:,[0,1,2]] = x[:,:,[2,1,0]]

    # transpose to channels first
    x = x.transpose((2, 0, 1))
    x = np.expand_dims(x, axis =0)
    x = x / 255
    pred_class = np.argmax(model.predict(x))
    heatmap = visualize_cam(model, layer_idx, [pred_class], x)
    heatmaps.append(heatmap)

It seems as though the same issue occurs with saliency maps as well.

[raghakot]

With the latest commit, I separated the code that is responsible for overlaying heatmap onto image because the heatmap can be generated for N-dim images and overlay only makes sense for rgb or grayscale images.