search

hila-chefer / Transformer-Explainability

[CVPR 2021] Official PyTorch implementation for Transformer Interpretability Beyond Attention Visualization, a novel method to visualize classifications by Transformer based networks.
MIT License
1.75k stars 232 forks source link

visualize TimeSformer #24

Closed Holmes-Alan closed 3 years ago

Holmes-Alan commented 3 years ago

Hello, do you think that you can show one example of using your code for TimeSformer visualization?

hila-chefer commented 3 years ago

Hi @Holmes-Alan, thanks for your interest in our work! I’m not really familiar with TimeSformer, so adding it to this repo isn’t planned. If the implementation is similar to either ViT or BERT, you can use the LRP layers implemented in this project. Otherwise, we have a follow up work that just got accepted to ICCV which eliminates the use of LRP making it significantly simpler to implement.

I’ll be happy to help with any other questions regarding the implementation itself. Best, Hila.

mohamedmansour0 commented 3 years ago

@Holmes-Alan have you found a way to visualize the Timesformer attention?

Holmes-Alan commented 3 years ago

no, I am not sure how to do it.