search

idiap / fullgrad-saliency

Full-gradient saliency maps
Other
201 stars 31 forks source link

FullGrad for Vision Transformers #14

Open imanuelroz opened 2 years ago

imanuelroz commented 2 years ago

Hi I wanted to know if there is a version of FullGrad which could be applied on Vision Transformers like ViT or the Swin Transformer, or if there are some small changes that could be done in the code in order to do it. Thank you in advance.

suraj-srinivas commented 2 years ago

Hi! Sorry for the late reply. So technically Fullgrad is proposed for convolution or fully connected neural networks, and as a result the completeness conditions may not be satisfied for transformers.

However you are free to use Simple / Smooth FullGrad in this case, which don't have completeness associated. I haven't tested it for transformers myself, but you'd need to change this line: https://github.com/idiap/fullgrad-saliency/blob/2121d212494d8dc401e27ec8198551efe68dd58f/saliency/tensor_extractor.py#L25 to include self-attention layers and exclude fully connected maybe.

If you do happen to use it, I'd be happy to learn about your experience or the issues you faced!