Question about Layer-wise Relevance Propagation

[hojin4671]

Hi, thanks for providing nice toolbox. I have one question about "layer-wise relevance propagation (LRP)" method in the code. It seems like it was implemented based on the equation [7] in the paper (Samek, W., Binder, A., Montavon, G., Bach, S., & Müller, K. R. (2015). Evaluating the visualization of what a deep neural network has learned. arXiv preprint arXiv:1509.06321.).

In the code, following that dzdy = res(i+1).dzdx ./ (res(i+1).x + eps), dzdy is put into convolution backpropagation, then run res(i).dzdx = dzdx .* res(i).x. But I am wondering how z_ij can be same with res(i).x here. Also, I'm wondering how the conservation principle is maintained. Actually, I am trying to implement alpha-beta version of LRP. Could you let me know the intuition of the code about this part in detail? Thanks.

[hojin4671]

Now I understand. By multiplying dzdx with res(i).x, it becomes z_ij. It's so cool! Thanks again.

[FelixGruen]

Hi,

the lines you are quoting (btw, you can link to specific lines on GitHub directly to make it easier for others to find the relevant code. I edited your question accordingly) are acutally based on equation (6) in the paper. Or more specifically on equations (50), (51), (56), (58), and (62) in Bach et al. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.

For a convolutional layer without the activation function as it is in MatConvNet the activation a_j (or x_j in the Bach paper) is equal to the weighted sum of the inputs

a_j = z_j = \sum_i a_i w_{ij} + b_j (standard conv layer, equations (50) and (51) in the Bach paper)

So just by replacing the terms in equation (62) you'll find

R^l_i
= \sum_j R^{l, l+1}_{i \leftarrow j}
= \sum_j \frac{a_i w_{ij}}{z_j} \cdot R^{l+1}_j
= \sum_j \frac{a_i w_{ij}}{a_j} \cdot R^{l+1}_j
= a_i \sum_j w_{ij} \cdot \frac{R^{l+1}_j}{a_j}

(btw, if there is a way to typeset LaTeX in a comment, I'd be grateful)

which is basically the equation for the backpropagation of gradients except for two small differences:

• before the backpropagation step all values are divided by the activation of their corresponding unit after the convolutional layer: \frac{R^{l+1}_j}{a_j} in the last line above and dzdy = res(i+1).dzdx ./ (res(i+1).x + eps) in code

• after the backpropagation step, all values are multiplied by the activation of their corresponding unit before the convolutional layer: a_i in the last line above and res(i).dzdx = dzdx .* res(i).x in code

For the conservation properties see equations (55) and (59) in the paper.

I hope this answers your questions and explains the inutition behind the code.

[hojin4671]

Thanks for the reply. It's really helpful!