use torch-cam/scripts/cam_example.py throw a exception:

Ethan-Chen-plus commented 1 year ago

Bug description

I clone and install torchcam,and then I use:

!python torch-cam/scripts/cam_example.py \
        --img test_img/border-collie.jpg \
        --savefig output/B1_border_collie.jpg \
        --arch resnet18 \
        --class-idx 232 \
        --rows 2

Code snippet to reproduce the bug

python torch-cam/scripts/cam_example.py \
        --img test_img/border-collie.jpg \
        --savefig output/B1_border_collie.jpg \
        --arch resnet18 \
        --class-idx 232 \
        --rows 2

Error traceback

WARNING:root:no value was provided for `target_layer`, thus set to 'layer4'.
WARNING:root:no value was provided for `fc_layer`, thus set to 'fc'.
WARNING:root:no value was provided for `target_layer`, thus set to 'layer4'.
WARNING:root:no value was provided for `target_layer`, thus set to 'layer4'.
WARNING:root:no value was provided for `target_layer`, thus set to 'layer4'.
WARNING:root:no value was provided for `target_layer`, thus set to 'layer4'.
WARNING:root:no value was provided for `target_layer`, thus set to 'layer4'.
WARNING:root:no value was provided for `target_layer`, thus set to 'layer4'.
WARNING:root:no value was provided for `target_layer`, thus set to 'layer4'.
WARNING:root:no value was provided for `target_layer`, thus set to 'layer4'.
Traceback (most recent call last):
  File "/mnt/k/kewei/haahsummer/torch-cam/scripts/cam_example.py", line 146, in <module>
    main(args)
  File "/mnt/k/kewei/haahsummer/torch-cam/scripts/cam_example.py", line 91, in main
    activation_map = extractor(class_idx, scores)[0].squeeze(0).cpu()
  File "/mnt/k/kewei/haahsummer/torch-cam/torchcam/methods/core.py", line 163, in __call__
    return self.compute_cams(class_idx, scores, normalized, **kwargs)
  File "/mnt/k/kewei/haahsummer/torch-cam/torchcam/methods/core.py", line 187, in compute_cams
    weights = self._get_weights(class_idx, scores, **kwargs)
  File "/mnt/k/kewei/haahsummer/torch-cam/torchcam/methods/gradient.py", line 279, in _get_weights
    out = self.model(noisy_input)
  File "/home/kewei/miniconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1128, in _call_impl
    result = forward_call(*input, **kwargs)
  File "/home/kewei/miniconda3/lib/python3.9/site-packages/torchvision/models/resnet.py", line 283, in forward
    return self._forward_impl(x)
  File "/home/kewei/miniconda3/lib/python3.9/site-packages/torchvision/models/resnet.py", line 274, in _forward_impl
    x = self.layer4(x)
  File "/home/kewei/miniconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1131, in _call_impl
    hook_result = hook(self, input, result)
  File "/mnt/k/kewei/haahsummer/torch-cam/torchcam/methods/gradient.py", line 50, in _hook_g
    self.hook_handles.append(output.register_hook(partial(self._store_grad, idx=idx)))
  File "/home/kewei/miniconda3/lib/python3.9/site-packages/torch/_tensor.py", line 397, in register_hook
    raise RuntimeError("cannot register a hook on a tensor that "
RuntimeError: cannot register a hook on a tensor that doesn't require gradient

Environment

torch 1.11.0+cu102 torchcam 0.4.0.dev0

frgfm commented 1 year ago

Hello @Ethan-Chen-plus :wave:

Thanks for reporting this! I just investigated and here is what happened:

in order to save memory for users, I started removing unnecessary gradient computation in scripts
now this library uses pytorch hooks, and some hooks (including the ones I use to store gradients) throw errors if the output tensor doesn't have requires_grad set to True
so far so good, because for each input tensor I set its requires_grad to True (we avoid the grad computation for each model's parameter that way, which can save a lot of memory depending on the model size)
however, after investigating your problem, I realized the script was failing on SmoothGradCAMpp which is a bit specific. It creates a noisy input that will be forwarded into the model. And I had forgotten to set the grad computation there (until now, it couldn't crash because everything else was requiring gradient so the hook couldn't crash)

As this is pretty simple to fix, I just opened a PR (#204) to address this! This should be solved on the "main" branch within a few minutes :)

Let me know if you have any questions!

Ethan-Chen-plus commented 1 year ago

Good! Thank you for solving this!

zhangzw12319 commented 1 year ago

Thank you for solving this problem! However in current version(2023-02-05), this bug is not fixed in current main branch, and thus I've spent several hours debugging until coming across this issue. Therefore it is recommended that this small bug be fixed as soon as possible to help new rookies to save time. THX a lot.

frgfm commented 1 year ago

Hi @zhangzw12319 :wave:

What do you mean? About the specific bug mentioned in this very issue, I merged the PR fixing it 3 weeks ago and I just tried again the exact same snippet, it works correctly :thinking: Perhaps it's because you're using the last published version of the library? Does the problem persist if you install it in developer mode first?

git clone https://github.com/frgfm/torch-cam.git
pip install -e torch-cam/.

If you're referring to another bug or another snippet, this might be unrelated and in this case, feel free to open another issue :)

zhangzw12319 commented 1 year ago

Thank you for your response! I remember that I installed it using pip install torchcam at first glance. So it is because the last published version of the library. Hope that the published version will be fixed soon :)

在 2023-02-06 00:50:24，"F-G Fernandez" @.***> 写道：

Hi @zhangzw12319👋

What do you mean? About the specific bug mentioned in this very issue, I merged the PR fixing it 3 weeks ago and I just tried again the exact same snippet, it works correctly 🤔 Perhaps it's because you're using the last published version of the library? Does the problem persist if you install it in developer mode first?

git clone https://github.com/frgfm/torch-cam.git

pip install -e torch-cam/.

If you're referring to another bug or another snippet, this might be unrelated and in this case, feel free to open another issue :)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

frgfm / torch-cam