Closed nkoenen closed 2 years ago
Hey Niklas,
the problem is that you try to register the rule to the Sequential
model itself, instead of the Linear layer, which will not work. I'm actually surprised this does not crash.
Here's two ways to do it, one with only a single layer:
which prints:
and one with the full Sequential, using a custom Composite and Attributor:
which prints:
which is both what you computed by hand.
Have a look at the documentation, where there is also an example with only a single layer.
Awesome! Thank you for the quick reply.
Hi,
I've been working very extensively with the LRP method lately and I also tried to implement the method with the most commonly used rules by myself. In order to check the correctness of my implementation, I compared some results with already existing implementations (like yours ;) ). Thereby I always get different relevances with the AlphaBeta-rule (I already opened an issue in innvestigate with the same example). Maybe you can explain the following behavior or confirm that this is really a bug on your side:
Let's suppose we have only one layer with two inputs, one output and no activation function. The layer has the following weights and bias vector: W = (1, -1) and b = -1. For the input x = (1,1), the formula of the AlphaBeta rule (Eq. (60)) in Bach et al. reduces to (in this case is r_out = -1)
This yields a relevance of (-1, 0) for the Alpha1_Beta0-rule and (-2, 0.5) for the Alpha2_Beta1-rule. But with your implementation I get both times (0.5, -0.5). Also for other choices of alpha and beta I always get the same result. Here is my code snippet for the Alpha1_Beta0 rule:
I hope you can help me to clarify this behavior.
Best Niklas