search

hila-chefer / Transformer-Explainability

[CVPR 2021] Official PyTorch implementation for Transformer Interpretability Beyond Attention Visualization, a novel method to visualize classifications by Transformer based networks.
MIT License
1.75k stars 232 forks source link

About relprop in BERT.py #46

Closed szubing closed 2 years ago

szubing commented 2 years ago

Hi, Hila, I have a question about relprop in BertModel class in BERT.py: def relprop(self, cam, kwargs): cam = self.pooler.relprop(cam, kwargs)

print("111111111111",cam.sum())

    cam = self.encoder.relprop(cam, **kwargs)
    # print("222222222222222", cam.sum())
    # print("conservation: ", cam.sum())
    return cam

Why we do not need to reprop the embedding module by self.embeddings.relprop(cam, **Kwargs) here before returning cam?

hila-chefer commented 2 years ago

Hi @szubing thanks for your interest!

If I understand your question correctly, this is used for the full LRP baseline where the relevance is propagated all the way back to the input. Our method only requires propagation of relevance to the attention layers.

I hope this helps, but if not do let me know and I’ll do my best to clarify.

szubing commented 2 years ago

Hi @szubing thanks for your interest!

If I understand your question correctly, this is used for the full LRP baseline where the relevance is propagated all the way back to the input image. Our method only requires propagation of relevance to the attention layers.

I hope this helps, but if not do let me know and I’ll do my best to clarify.

Hi, Hila, thank you very much!