search

hila-chefer / Transformer-Explainability

[CVPR 2021] Official PyTorch implementation for Transformer Interpretability Beyond Attention Visualization, a novel method to visualize classifications by Transformer based networks.
MIT License
1.76k stars 234 forks source link

Could you add the baseline Grad-CAM implementation? #6

Closed ramprs closed 3 years ago

ramprs commented 3 years ago

Hi,

This work is really interesting. In the paper I saw that you had compared your approach with Grad-CAM among other approaches. However I was unable to find its implementation in the baselines folder. Is there any chance you could add this code?

Thanks in advance

hila-chefer commented 3 years ago

Hi @ramprs, thanks for your interest in our work! I refer you to: https://github.com/hila-chefer/Transformer-Explainability/blob/main/baselines/ViT/ViT_explanation_generator.py#L59 this is the implementation of the Grad-CAM baseline. You can also look at our segmentation tests to get an example of how to use this code with ViT: https://github.com/hila-chefer/Transformer-Explainability/blob/main/baselines/ViT/imagenet_seg_eval.py#L217.

Also, thanks for inventing Grad-CAM for all of us to use! I just wanted to kindly remind that our version of Grad-CAM is an adjustment of your original paper.

I hope this helps, and please feel free to ask for clarifications if needed.