hila-chefer / Transformer-Explainability

[CVPR 2021] Official PyTorch implementation for Transformer Interpretability Beyond Attention Visualization, a novel method to visualize classifications by Transformer based networks.
MIT License
1.75k stars 232 forks source link

LRP for mean #65

Open LokeshBadisa opened 7 months ago

LokeshBadisa commented 7 months ago

Hi Hila,

Outstanding Work!! I'm thinking to implement LRP for mean & sum function. How would you suggest implementing it? I'm thinking to use Linear layer if I know the dimensions beforehand. Is there any better way to do it? Do you suggest variants for sum & mean over elements of same tensor and sum and mean of 2 tensors?