Closed shaheen-syed closed 3 years ago
The guided gradients have a misleading effect on attribution map. It implies a heavy contribution of the raw input into the construction of the attribution map, and can cause to produce unexpected "valid" feature map for some classes.
A nice paper that is going though this effect (and others) is Sanity Check for Saliency Maps.
Thanks for letting me know. After I posted the issue I also come across the paper you mentioned, and some others addressing similar issues/concerns.
Closing this issue as it is more related to the method than the library.
I have a trained 4 class CNN model based on a VGG16 architecture. I am using gradcam to inspect where in each image each class activates. However, when I use gradcam with
use_guided_grads = True
, I get the same heatmap for each class_index. I expect the activations to be different for class_index 0, 1, 2 and 3. When I set the use_guided_grads to False, then the output is as expected, that is, a different heatmap for each class. I am attaching two plots for illustrative purposes. Is this expected behaviour or am I missing something?Activations of the same image for classes 0, 1, 2, 3 and![with_guided_grad](https://user-images.githubusercontent.com/9527819/88171022-ef0e5300-cc1e-11ea-85fd-1a12a3981f4d.png)
use_guided_grads = True
Activations of the same image for classes 0, 1, 2, 3 and![without_guided_grad](https://user-images.githubusercontent.com/9527819/88171038-f3d30700-cc1e-11ea-98da-c287008bf8b8.png)
use_guided_grads = False