AxisError: axis 2 is out of bounds for array of dimension 2

[asfandyarazhar13]

Getting the following error when trying out the cam function on an image example. This might be an issue with how I have loaded in my data, but not sure how to debug it.

Code to reproduce:

from pytorch_grad_cam import GradCAM, ScoreCAM, GradCAMPlusPlus, AblationCAM, XGradCAM, EigenCAM
from pytorch_grad_cam.utils.image import show_cam_on_image
import PIL

target_layers = [model.linear_layers[-1]]
img, label, path = next(iter(test_loader))
img, label = img.to(DEVICE), label.to(DEVICE)

img = img.float()

cam = GradCAM(model=model, target_layers=target_layers)

target_category = None

grayscale_cam = cam(input_tensor=img)

grayscale_cam = grayscale_cam[0, :]
visualization = show_cam_on_image(rgb_img, grayscale_cam, use_rgb=True)

Note the image is of shape torch.Size([64, 1, 128, 128]) Here is the traceback:

---------------------------------------------------------------------------
AxisError                                 Traceback (most recent call last)
<ipython-input-188-441d284cf4e0> in <module>()
     14 target_category = None
     15 
---> 16 grayscale_cam = cam(input_tensor=img)
     17 
     18 grayscale_cam = grayscale_cam[0, :]

7 frames
/usr/local/lib/python3.7/dist-packages/pytorch_grad_cam/base_cam.py in __call__(self, input_tensor, targets, aug_smooth, eigen_smooth)
    183 
    184         return self.forward(input_tensor,
--> 185                             targets, eigen_smooth)
    186 
    187     def __del__(self):

/usr/local/lib/python3.7/dist-packages/pytorch_grad_cam/base_cam.py in forward(self, input_tensor, targets, eigen_smooth)
     93         cam_per_layer = self.compute_cam_per_layer(input_tensor,
     94                                                    targets,
---> 95                                                    eigen_smooth)
     96         return self.aggregate_multi_layers(cam_per_layer)
     97 

/usr/local/lib/python3.7/dist-packages/pytorch_grad_cam/base_cam.py in compute_cam_per_layer(self, input_tensor, targets, eigen_smooth)
    128                                      layer_activations,
    129                                      layer_grads,
--> 130                                      eigen_smooth)
    131             cam = np.maximum(cam, 0)
    132             scaled = scale_cam_image(cam, target_size)

/usr/local/lib/python3.7/dist-packages/pytorch_grad_cam/base_cam.py in get_cam_image(self, input_tensor, target_layer, targets, activations, grads, eigen_smooth)
     52                                        targets,
     53                                        activations,
---> 54                                        grads)
     55         weighted_activations = weights[:, :, None, None] * activations
     56         if eigen_smooth:

/usr/local/lib/python3.7/dist-packages/pytorch_grad_cam/grad_cam.py in get_cam_weights(self, input_tensor, target_layer, target_category, activations, grads)
     20                         activations,
     21                         grads):
---> 22         return np.mean(grads, axis=(2, 3))

<__array_function__ internals> in mean(*args, **kwargs)

/usr/local/lib/python3.7/dist-packages/numpy/core/fromnumeric.py in mean(a, axis, dtype, out, keepdims)
   3371 
   3372     return _methods._mean(a, axis=axis, dtype=dtype,
-> 3373                           out=out, **kwargs)
   3374 
   3375 

/usr/local/lib/python3.7/dist-packages/numpy/core/_methods.py in _mean(a, axis, dtype, out, keepdims)
    145 
    146     is_float16_result = False
--> 147     rcount = _count_reduce_items(arr, axis)
    148     # Make this warning show up first
    149     if rcount == 0:

/usr/local/lib/python3.7/dist-packages/numpy/core/_methods.py in _count_reduce_items(arr, axis)
     64     items = 1
     65     for ax in axis:
---> 66         items *= arr.shape[mu.normalize_axis_index(ax, arr.ndim)]
     67     return items
     68 

AxisError: axis 2 is out of bounds for array of dimension 2

[jacobgil]

Hi, I'm looking into this.

Can you please confirm that there is an 1 in the shape, The shape is : torch.Size([64, 1, 128, 128]) Not torch.Size([64, 128, 128])

Right?

[asfandyarazhar13]

Thanks for the response! Just double checked and the shape actually torch.Size([16, 1, 128, 128]). However, still 4 dims.

[jacobgil]

I need to reproduce this, this might take me another 2-3 days to look at it. I will update.

[asfandyarazhar13]

Appreciate it. Looking forward to your response. Note when I try to reshape the image to torch.Size([16, 128, 128]), the following error occurs:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-43-6d82ff2c42c8> in <module>()
     13 print(img.shape)
     14 
---> 15 grayscale_cam = cam(input_tensor=img)
     16 
     17 grayscale_cam = grayscale_cam[0, :]

11 frames
/usr/local/lib/python3.7/dist-packages/pytorch_grad_cam/base_cam.py in __call__(self, input_tensor, targets, aug_smooth, eigen_smooth)
    183 
    184         return self.forward(input_tensor,
--> 185                             targets, eigen_smooth)
    186 
    187     def __del__(self):

/usr/local/lib/python3.7/dist-packages/pytorch_grad_cam/base_cam.py in forward(self, input_tensor, targets, eigen_smooth)
     72                                                    requires_grad=True)
     73 
---> 74         outputs = self.activations_and_grads(input_tensor)
     75         if targets is None:
     76             target_categories = np.argmax(outputs.cpu().data.numpy(), axis=-1)

/usr/local/lib/python3.7/dist-packages/pytorch_grad_cam/activations_and_gradients.py in __call__(self, x)
     40         self.gradients = []
     41         self.activations = []
---> 42         return self.model(x)
     43 
     44     def release(self):

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

<ipython-input-28-838fcb9d22d5> in forward(self, x)
     32 
     33     def forward(self, x):
---> 34         x = self.conv(x)
     35         x = self.fc(x)
     36         return x

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/container.py in forward(self, input)
    139     def forward(self, input):
    140         for module in self:
--> 141             input = module(input)
    142         return input
    143 

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/container.py in forward(self, input)
    139     def forward(self, input):
    140         for module in self:
--> 141             input = module(input)
    142         return input
    143 

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py in forward(self, input)
    444 
    445     def forward(self, input: Tensor) -> Tensor:
--> 446         return self._conv_forward(input, self.weight, self.bias)
    447 
    448 class Conv3d(_ConvNd):

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight, bias)
    441                             _pair(0), self.dilation, self.groups)
    442         return F.conv2d(input, weight, bias, self.stride,
--> 443                         self.padding, self.dilation, self.groups)
    444 
    445     def forward(self, input: Tensor) -> Tensor:

RuntimeError: Expected 4-dimensional input for 4-dimensional weight [32, 1, 3, 3], but got 3-dimensional input of size [16, 128, 128] instead

[asfandyarazhar13]

@jacobgil Hi again, just wanted to kindly follow up on this. Any luck?

[jacobgil]

Hi, what is 16 here, Is it a batch size or the image color channels?

Does the model run on 16 color channels ?

[jacobgil]

If it's the number of color channels, can you please try reshaping the input tensor into 1 x 16 x 128 x 128 ?

[asfandyarazhar13]

16 is the batch size. So should I still try to reshape it to (1, 16, 128, 128)?

[jacobgil]

No, I just wanted to make sure. So there is a batch of 16 different images we want to compute the CAM for, each is "grayscale" with only one color channels?

[asfandyarazhar13]

yes exactly

[jacobgil]

Ok thanks. I'm looking into it right now, will update.

[jacobgil]

I wrote a unit test to reproduce this, but couldn't reproduce the error so far. I modified a resnet18 model accept 1 color channel to test this.

Can you give a pointer on what model you're using - and what target layers? So I can test with it or something similar.

[jacobgil]

I just notice something - What is linear_layers ?

The target layers used need to have 2D outputs. linear_layers is a suspicious name so I want to make sure:)

[asfandyarazhar13]

Ah yes, that checks out. Thanks for the catch, it is working now!

[asfandyarazhar13]

Appreciate your time and effort :) Should've seen this myself!

[Dobatymo]

I had the same error. Just for reference in my case the target layer was not set to trainable which caused this.

[franeknowak]

I had the same error. Just for reference in my case the target layer was not set to trainable which caused this.

Many thanks Dobatymo, this has helped me a lot!

[arushsharma24]

Thanks a lot guys. I had the same error and I was facing this error due to choosing the wrong target_layers as well. I chose a 'classifier' layer in the EfficientNet_B0 model, and choosing the avgpool layer worked.

[wujiang0156]

I had the same error , Did someone fix it? thanks

[Girik-Khullar]

@arushsharma24 @Dobatymo Thx for the comment here. Finally got gradcam working.