Closed Patrick-Ni closed 2 weeks ago
Hey,
you could look at this issue https://github.com/rachtibat/LRP-eXplains-Transformers/issues/2.
You can't simply take the gradients at the hidden_states, because PyTorch does not save gradients on them.
You must either activate gradients for these tensors by modifying the model source code writing at the position you want to record hidden_states.retain_grad()
or use backward hooks.
Keep in mind, that LXT does nothing crazy! It is as if you would work with a normal PyTorch model. All tools of PyTorch work like always.
I would recommend that you read more about backward hooks and the retain_grad attribute.
Hope it helps
added example for obtaining relevance at the residual stream at https://lxt.readthedocs.io/en/latest/feature-visualization.html#latent-feature-attribution
Hey @rachtibat : Thank you for your excellent work. I have one last question. I want to understand the contribution of the hidden states from each layer to the final result, not just the input embeddings. I tried to understand the code and the paper, and then made the following modifications:
But this does not work. I would like to ask how can I achieve this?