Closed Holt59 closed 4 years ago
Hi @Holt59, thank you for the question. That's interesting! I'll debug it. In terms of the model state: Those are the hooks that perhaps aren't getting removed. We actually recently fixed it so that we remove the hooks for all cases. For that you might need github version.
Hi @Holt59. While @NarineK looks into the main issue, I thought to refer you to https://github.com/pytorch/captum/issues/370 regarding re-using the model after an error occurs. The new error is likely related to dangling hooks and is fixed in master as explained in the comments. Hope this helps
@Holt59, I've been debugging this issue. There seem to be some inconsistencies in the backward pass. In the meanwhile, as a workaround, if you want to attribute to the inputs of the MaxPool2D layer it will work. By default we attribute to the outputs of the layer.
model = torchvision.models.vgg16(pretrained=True)
u = captum.attr.LayerDeepLift(
model, list(model.features.children())[23]).attribute(
torch_im[None, ...], target=156, attribute_to_layer_input=True)[0]
Actually what you were doing will be equivalent to:
u = LayerDeepLift(
model, list(model.features.children())[24]).attribute(
torch_im, target=156, attribute_to_layer_input=True)[0]
as a workaround
@Holt59 , did that workaround work for you?
The workaround seems to work but I cannot use it in my code base like this since I am trying to compute attributions for multiple layers (and I don't know the following layer). But that's not a big issue, I'm not particularly interested in the MaxPool layers, I can leave them out.
@Holt59, this PR #390 will fix the problem with MaxPool. To give more context, this problem happened because in the forward_hook
we return cloned output tensor and that made the MaxPool modules complex. Since there is a bug related to complex modules in PyTorch and backward_hook
, that is, returned input gradients represent only a subset of inputs, it wasn't able to compute the multipliers correctly.
More details about the issue can be found here: https://pytorch.org/docs/stable/nn.html#torch.nn.Module.register_backward_hook
Another point that I wanted to bring up is: In VGG the modules might get reused (you might want to check that). We want to make sure that this isn't happening for the layer algorithms and DeepLift. If the activations get reused. You can simply redefine the architecture (that's easy to do). More info about it can be found here:
https://github.com/pytorch/captum/issues/378#issuecomment-633309752
This got fixed through: #390
I am trying to use
LayerDeepLift
on multiple layers of a VGG16 model fromtorchvision.models
. It works for all layers exceptMaxPooling2D
layers.The following (layer
23
is aMaxPool2d
layer):Raises the following:
It works on all layers except the
MaxPool2d
layers ofvgg16.features
(it works with the average pooling layer).I am not sure if this is a restriction of DeepLift or an error in the implementation?
Also, when the error occurs, the model seems to be left in some weird state as re-using it leads to
IndexError: tuple index out of range
(even with a brand newcaptum.attr.LayerDeepLift
instance).