fchollet / deep-learning-with-python-notebooks

Jupyter notebooks for the code samples of the book "Deep Learning with Python"
MIT License
18.68k stars 8.65k forks source link

Issue with Class Activation Map" (CAM) visualization - zero mean intensity of gradient #73

Open HRKpython opened 6 years ago

HRKpython commented 6 years ago

I am using Keras with tensorflow backend and I have fine-tuned the last Conv layer and FC layer of my network based on VGG weights. Now I am using CAM technique to visualize which parts of my image triggered the prediction and I get all zeros for mean intensity of the gradient over a specific feature map channel.

I have 4 classes, for my test sample these are the prediction:

preds_sample = model.predict(x)
output>> array([[1., 0., 0., 0.]], dtype=float32)

# This is the "sample image" entry in the prediction vector    
image_0 = model.output[:, 0]

last_conv_layer = model.get_layer('conv2d_13')
grads = K.gradients(toilet_w, last_conv_layer.output)[0]
grads.shape
output>> TensorShape([Dimension(None), Dimension(512), Dimension(14), Dimension(14)])

Since I am using theano image ordering - when I calculate the mean of grads my axis is (0,2,3)

from keras import backend as K
K.set_image_dim_ordering('th')

pooled_grads = K.mean(grads, axis=(0,2,3))
pooled_grads.shape
output>> TensorShape([Dimension(512)])

iterate = K.function([model.input], [pooled_grads, last_conv_layer.output[0]])
pooled_grads_value, conv_layer_output_value = iterate([x])
pooled_grads_value.shape, conv_layer_output_value.shape
output>> ((512,), (512, 14, 14))

pooled_grads_value is all zero. Any thoughts/help appreciated.

ochrosiaCoccineaBullet commented 4 years ago

Have you solved the problem?I also had problems with the grad-cam.