search

haofanwang / Score-CAM

Official implementation of Score-CAM in PyTorch
MIT License
399 stars 66 forks source link

2 versions of ScoreCAM #17

Closed zcuncun closed 3 years ago

zcuncun commented 3 years ago

There are 2 versions of ScoreCAM on arXiv. And the implementation of other libraries is also different. The difference is the use of softmax operation and subtraction of baseline. Which version is better?

haofanwang commented 3 years ago

@zcuncun, thanks for your interests.

Usually, we set the baseline to a black image (all zeros), and all experiments in our paper use this setting. Regarding the use of Softmax operation, can you specifically explain the difference?

I don't know which implementation you use, different implementations may be slightly different.

zcuncun commented 3 years ago

In V1, the output score Sk is taken as the weight of the k_th activation map. Sk=Softmax(F(Mk)) Wk = Sk

In V2, a Softmax operation is used on the CIC score Sk=Softmax(F(Mk)) - Softmax(F(Mbaseline)) Wk = Exp(Sk) / sum(Exp(Si))

haofanwang commented 3 years ago

They are actually the same thing. In our experiment or code, we just set the baseline image to a black image. Then V2 degrades to V1. As the S_k in V1 is already within range [0,1], we don't need to do normalization. Hope this helps.

zcuncun commented 3 years ago

Thanks, you do help me a lot!

Technically, they are not the same thing. If you are concerned, check following repos which reimplement your work. They use softmax at different places and give different results.

https://github.com/jacobgil/pytorch-grad-cam/blob/c009adac84ec83565c88c692a13bd3581e7c24eb/pytorch_grad_cam/score_cam.py#L49

https://github.com/frgfm/torch-cam/blob/9c94bd4365b4dbb92055802ce87e717bbf89d9e3/torchcam/cams/cam.py#L174