GradCAM opposite visualization

[bobr96]

Hi, I implemented a custom CNN for binary classification with following structure. The goal is to check if a specific object is present (class 1) or not present (class 0) in the image. The trained CNN does have very good results on validation and test dataset. It is not overfitted.

• conv2d (Conv2D) (None, 100, 100, 16)
• max_pooling2d (MaxPooling2D) (None, 50, 50, 16)
• conv2d_1 (Conv2D) (None, 50, 50, 32)
• max_pooling2d_1 (MaxPooling2 (None, 25, 25, 32)
• conv2d_2 (Conv2D) (None, 25, 25, 64)
• max_pooling2d_2 (MaxPooling2 (None, 12, 12, 64)
• conv2d_3 (Conv2D) (None, 12, 12, 128)
• max_pooling2d_3 (MaxPooling2 (None, 6, 6, 128)
• global_average_pooling2d (Gl (None, 128)
• dense (Dense) (None, 64)
• dense_1 (Dense) (None, 2)

Now I want to visualize the output of the filter "conv2d_3" for the class "1" with GradCAM. For this I used the source code from this awesome library.

explainer = tf_explain.core.grad_cam.GradCAM() output = explainer.explain(data, model=my_model, class_index=1, layer_name='conv2d_3', use_guided_grads=False, image_weight=0.5, colormap=cv2.COLORMAP_JET)

With model.predict() the correct result for class "1" is returned: [0.01, 0.99].

However, GradCAM marks for class "1" (class_index=1) exactly the opposite as the specific object. So everything except the object is marked. If I analyze class "0" instead (class_index=0), the specific object is marked correctly. Strangely enough, this is the case with every image and only if use_guided_grads=False. It seems like class 0 and 1 have been swapped when calculating the Grad-CAM. If use_guided_grads=True, both classes show the same activations.

Do you have any idea why the visualization shows exactly the opposite to the prediction? If you need more information, I can send it to you..

[RaphaelMeudec]

Could you add the figures to the issue?

[bobr96]

These are the visualizations created by GradCAM. Actually, I expect the two images on the right side to be swapped. So class_idx=0 should analyze only the background and class_idx=1 only the object in the middle.

What am I doing wrong? Thank you very much for your help.

[RaphaelMeudec]

The last one is indeed very odd. What is the scale of the figure? (min and max values)

Regarding the fact that your network is always looking at the center: this is probably normal. You're passing an image with the object in it, hence the interpretation just says: I gave a low value for class_index=0 because I've seen this region in the middle. The background is not very informative, hence most of the information is about the center region.

[bobr96]

I have trained the network with 100x100 images with values in the interval [0,1].

Yes, it is normal that the objects are always in the center. But that's why I don't understand this visualization. Here, for the predicted class (1), the center is ignored. It is the image on the lower right side. Here, only the background except the center is examined (=red area). Or maybe I interpret it wrong?

[bobr96]

I reproduced this behavior with a different implementation. For this I used the source code from the official documentation of Keras (Link). So it is obviously not a problem with your implementation. However, I still don't understand the interpretation. Can you help me to understand it?

These are the activations of the most interesting layers. The red areas are the most important ones. The blue areas, on the other hand, are the unimportant ones. In the first two layers, the object is detected correctly. In "conv2d_2", however, everything except the object is recognized. How would you interpret this shift in attention?

To clarify: The result "class 1" in "dense_1" is correct. But it seems to me that the object in the middle is not important for this final decision?

Or is a possible interpretation that the CNN recognizes the object in "max_pool" and therefore looks at the background to check what the background is doing? Thanks for your help.

[RaphaelMeudec]

This is weird, could you generate the output at each conv layer? So we can compare the heatmap with the generated output from each convolution/maxpool?

[bobr96]

Yes, of course. I have created binaries (.npy) for each layer. I hope this helps with troubleshooting. layers.zip

[bobr96]

Here are the activations of each layer

[RahimiPhz]

Hi, I am facing the same problem of having opposite visualization, has this been resolved?

Thank you.