MikiFER
commented
8 months ago Hi @chr5tphr is there any news on this bug? I noticed that when setting zero_params='bias' for EpsilonPlusFlat composite total attribution sums to ~1 but sometimes negative attribution may occur and that negative attribution sometimes is small but sometimes it accounts to ~-1 and more so its significant and positive part of the attribution in that case accounts to almost ~2 to negate the effect.
Here is code I used to replicate this issue (multiple runs may be required):
import torch
import torch.nn as nn
from torchvision.models import resnet34
from zennit.torchvision import ResNetCanonizer
from zennit.composites import EpsilonPlusFlat
import matplotlib.pyplot as plt
model = resnet34(weights=None)
# create a composite, specifying the canonizers
composite = EpsilonPlusFlat(canonizers=[ResNetCanonizer()], zero_params='bias')
target = torch.eye(1000)[[437]]
input_data = torch.rand(1, 3, 224, 224)
input_data.requires_grad = True
with composite.context(model) as modified_model:
output = modified_model(input_data)
attribution, = torch.autograd.grad(output, input_data, target)
relevance = attribution.cpu().sum(1).squeeze(0)
if torch.any(relevance < 0):
print(relevance[relevance < 0].sum())
print(relevance[relevance > 0].sum())
print(relevance.sum())
plt.imshow(relevance.numpy())
plt.show()If you maybe don't have the time to deal with this issue could you maybe point me in the right direction so I could try to fix it since it is critical for me to fix this issue in order to continue with my research.
chr5tphr
With the introduction of #185 , ResNet18 attributions result in negative attribution sums in the input layer, leading to bad attributions. Although #185 increased the stability of the attribution sums for ResNet, the previous instability seems to have inflated the positive parts of the attributions, circumventing this problem pre #185.
This seems to be related to leaking attributions (#193 ) combined with skip connections that can cause negative attributions.
A quickfix for
EpsilonGammaBoxis to use a slightly higher gamma value.