Closed gprabaha closed 7 years ago
I got similar constant heatmap (but mine are covered by a blue layer). #27 mentioned this problem as well and I think it's not fully fixed?
Can you guys try this again in latest? There were a couple of big changes with the latest commit.
@raghakot I'm using the latest version on PyPI (0.3.1).
from keras.preprocessing.image import ImageDataGenerator
from keras.models import load_model
batch_size = 2
test_datagen = ImageDataGenerator(rescale=1./255)
validation_generator = test_datagen.flow_from_directory(
'validation',
target_size=(200, 200),
batch_size=batch_size,
class_mode='binary',
shuffle=False)
model = load_model('weights.hdf5')
res_cnn = 1- best.predict_generator(validation_generator,steps=1)
res_cnn_class = [0 if x < 0.5 else 1 for x in res_cnn[:,0]]
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
from vis.utils import utils
from vis.visualization import visualize_cam
model = model
fig = plt.figure(figsize=(20,20))
i = -1
for fn in validation_generator.filenames:
i += 1
subplot = fig.add_subplot(1,12,i+1)
if i < 1:
subplot.set_title('Class A')
else:
subplot.set_title('Class B')
seed_img = utils.load_img('validation/'+fn, target_size=(200,200))
heatmap = visualize_cam(best, 7, [res_cnn_class[i]], seed_img)
plt.imshow(heatmap, cmap=plt.get_cmap('gray'))
plt.axis('off')
plt.show()
Attached zip file contains model weights, python notebook of the code, and test images to reproduce the problem. debug.zip
@raghakot I reinstalled keras-vis
from source, and the new error that is now showing up is,
Traceback (most recent call last):
File "keras-vis-cam.py", line 38, in <module>
heatmap = visualize_cam(model, layer_idx, [pred_class], seed_img)
File "/home/yx96/anaconda2/lib/python2.7/site-packages/keras_vis-0.3.1-py2.7.egg/vis/visualization.py", line 371, in v
isualize_cam
penultimate_output = penultimate_layer.output
AttributeError: 'numpy.ndarray' object has no attribute 'output'
It seems to be failing to get the output from the penultimate layer! This error is showing, of course, for all acceptable layer_name
, like, 'predictions'
, or 'block2_conv2'
.
Hi, The API has changed. There is now visualize_class_cam
and visualize_regression_cam
with visualize_cam
being more general that works on custom loss functions. You might want to use visualize_class_cam
now. API docs should be updated. I am still working on completely overhauling the docs and examples.
@raghakot Thanks for the update. After updating the package to 0.3.2 I'm able to import the new function visualize_class_cam. However now the output are pure blue squares (I just simply replaced visualize_cam with visualize_class_cam) :
It is very likely the case that the penultimate_layer has a spatial resolution of (1, 1). You might have to specify a pre Conv layer manually via penultimate_layer_idx
. I added examples for MNIST in examples/. Highly recommend checking that out to understand grad-CAM.
Is your code the same as this?
@raghakot The MNIST example runs well on my machine but my model still gets these blue squares even after I set the penultimate layer idx (now the idx corresponds to a 48x48x64 maxpooling layer) for CAM. visualize_class_saliency also gives me these blue squares. My input size is 200x200 and the output size of last maxpooling layer is 48x48x64. The code for both CAM and saliency map I used is included in a notebook in this compressed file, along with two test images and the model weights. debug.zip
@wanting0wang It appears that the gradients are all 0's with respect to the input. This could mean that your model is overfitting or not trained properly since none of the input pixels have an effect on the output. How was this trained? It also seems too shallow. You probably need more conv layers and reduce spatial resolution down to (7, 7) or (5, 5) before applying Dense layer.
Also, this is unrelated to MNIST dataset, so you should create a separate issue for it and avoid cluttering this thread. I will help with debugging on the new thread.
I am closing this due to lack of correspondence. Feel free to reopen the issue if you're still encountering problems.
@raghakot Hi sorry for the late reply.. I was on holiday. I think my problem is because of the high spatial resolution (and that increases the parameters a lot and leads to overfit). I trained another model with lower spatial resolution in the last maxpooling layer (about 9x9) and everything looks great. Thank you for the suggestions!
Great. Glad it worked out :)
I was trying out the class activation map of some cat and dog images for different layers of VGG16 (using
keras.applications.vgg16
, notvis.utils.vggnet
), and the code, shown in (https://raghakot.github.io/keras-vis/visualizations/attention/) does not seem to be working for layers other than 'predictions'. When I uselayer_name='block1_conv2'
, it is throwing an error,ValueError: Unable to detect penultimate 'Convolution2D' or 'Pooling2D' layer for layer_idx: 2
. On the other hand, forlayer_name='block2_conv2'
, orlayer_name='block3_conv2'
there is anAssertionError
forlen(item_slice) == 4
.Further, when I am using
layer_name='predictions'
, the prediction is performed correctly for the cat and dog image (Tabby, and Siberian Husky), but the heat map is barely on the dog.The problem is seen even for the images shown in the website, at least for the Tiger and the Spider Web.
When I use the VGG16 from
vis.utils.vggnet
, though, theheatmap
makes sense!Though, even in this case, when I change the layer to
fc1
orfc2
, the entirety of the image lights up.The code I am using is: