Layer-wise Relevance Propagation in PyTorch
Basic implementation of unsupervised Layer-wise Relevance Propagation (LRP, Bach et al., Montavon et al.) in PyTorch for VGG networks from PyTorch's Model Zoo. This tutorial served as a starting point. In this implementation, I tried to make sure that the code is easy to understand and easy to extend to other network architectures.
I also added a novel relevance propagation filter to this implementation resulting in much crisper heatmaps (see my blog for more information). If you want to use it, please don't forget to cite this implementation.
This implementation is already reasonably fast. It is therefore also suitable for projects that want to use LRP in real time. Using a RTX 2080 Ti graphics card I reach 53 FPS with the VGG-16 network.
If I find the time, I will provide a more model agnostic implementation. I also welcome pull requests improving this implementation.
You can find more information about this implementation on my blog.
Executive Summary
Running LRP for a PyTorch VGG network is fairly straightforward
import torch
from torchvision.models import vgg16, VGG16_Weights
from src.lrp import LRPModel
x = torch.rand(size=(1, 3, 224, 224))
model = vgg16(weights=VGG16_Weights.DEFAULT)
lrp_model = LRPModel(model)
r = lrp_model.forward(x)Example LRP Projects
Currently there are three minimal LRP projects. These projects can be executed with the following commands:
python -m projects.per_image_lrp.main
python -m projects.real_time_lrp.main
python -m projects.interactive_lrp.mainPer-image LRP
Per-image LRP applies Layer-wise relevance propagation to all images located in the input folder. With this option high resolution relevance heatmaps can be created.
 |
 |
 |
 |
 |
 |
 |
 |
Real-time LRP
Real-time LRP lets you use your webcam to create a heatmap video stream. The example below shows images filmed from a monitor.
Interactive LRP
Interactive LRP allows you to manipulate the input space with different kind of patches. This option is interesting to see how the network reacts to changes in the image input space.
| Input | Relevance Scores |
|---|---|
 |
 |
Relevance Filter
This implementation comes with a novel relevance filter for much crisper heatmaps. Examples show the $z^+$-rule without and with additional relevance filter.
 |
 |
 |
 |
TODOs
- Add support for other network architectures (model agnostic)
License
MIT
Citation
@misc{blogpost,
title={Layer-wise Relevance Propagation for PyTorch},
author={Fischer, Kai},
howpublished={\url{https://github.com/kaifishr/PyTorchRelevancePropagation}},
year={2021}
}