search

ramprs / grad-cam

[ICCV 2017] Torch code for Grad-CAM
https://arxiv.org/abs/1610.02391
1.48k stars 224 forks source link

Gradient values and gradient relevance #18

Open javiercoroneltum opened 5 years ago

javiercoroneltum commented 5 years ago

Hi, I've been trying the code for a while and have a couple of questions about the gradients. To give some context here, I would like to identify which feature maps are the most relevant for a specific class, e.g. not use all the feature maps for visualization, but only the most important.

So the paper says that after doing the back-propagation for a specific class, we average the gradients, and that captures the "importance" of a feature map. I've been exploring the distribution for each layer using alexnet and here I show the distribution of those averaged gradients for a specific layer in AlexNet: image So we have a distribution with both positive and negative gradients that are close to zero. As the code is implemented, we use all of those gradients and that results in a visualization that looks like this for the class gondola from imagenet: image At first, I thought, ok I won't use all the gradients, I only want the gradients closest to zero and so I set a window to include those. And here comes my first question:

  1. Are the gradients closest to zero (or zero), the ones that would require a smaller update for the kernels during training? meaning that those feature maps are the most relevant and meaningful for the final classification?

After trying that I didn't get an improved visualization, so I kept exploring the gradients. Then I decided to only use the positive or only the negative gradients, here are the results: Only positive gradients: image Only negative gradients: image It seems for me that the negative gradients are the most meaningful, even more than using all the gradients, and the same happens for other images as well. Here I have my next question:

  1. Why would the negative gradients be more relevant for the visualization?
  2. What's going on with only the positive values and why are those guiding to a completely different visualization than in the other cases?

Thanks in advance for any answer and I'm looking forward for the discussion.

huawei-lin commented 5 years ago

Hey. I have the same problem as you. Have you solved it？

MaxwellHogan commented 3 years ago

https://ieeexplore.ieee.org/document/8955509

this paper has came out later that year you guys posted, it presents BiGradV which is "based on the fact both positive and negative gradients can be sensitive to class discrimination"