search

hila-chefer / Transformer-Explainability

[CVPR 2021] Official PyTorch implementation for Transformer Interpretability Beyond Attention Visualization, a novel method to visualize classifications by Transformer based networks.
MIT License
1.75k stars 232 forks source link

Token Classification #28

Closed zooeygirl closed 2 years ago

zooeygirl commented 2 years ago

Hi there,

Thank you very much for putting this together. It is very cool!

I would like to adapt it to a Bert model that has a linear classifier on top (a class is predicted for each token in the input). I was wondering if you have any tips for the easiest way to do this?

Thanks a lot.

zooeygirl commented 2 years ago

Nevermind, I found a solution. But I have a related question: Are you planning to extend this to other language models (GPT-2 for example)?

zooeygirl commented 2 years ago

Hi again, So I am trying to adapt your work to GPT2, which is quite similar to BERT save a few differences. For example, where BERT uses a linear layer to project the query, key and value in its attention unit, GPT2 uses Conv1D layer. What is the best way to treat relprop for Conv1D? Thanks a lot.

hila-chefer commented 2 years ago

Hi @zooeygirl thanks for your interest in our work! currently, we are not planning on expanding to GPT-2. In order to avoid implementing LRP, you can consider using our second paper (from ICCV21') where we eliminated the use of LRP. If you're still interested in using LRP, here's a nice resource with LRP implementations.

Feel free to ask for clarifications if needed.

Best, Hila.

zooeygirl commented 2 years ago

Thank you for the response and for the resources, Hila. I will check them out. :-)