search

BioroboticsLab / IBA

Information Bottlenecks for Attribution
MIT License
75 stars 9 forks source link

Use of k=hwc for selecting beta #40

Open KellyWuuu opened 2 years ago

KellyWuuu commented 2 years ago

Hi! The paper is really great, and thank you for sharing the code here.

I have a quick question about how you calculate k. In the paper, it is said that k = hwc, where h is height, w is width and c is channels. However, when I referred to Appendix C for VGG-16, it seems that k needs to equal 1. Would you mind explaining the calculation of k and its use?

Thanks a lot!

berleon commented 2 years ago

Hi! Thank you for your interest in our paper. Also in the Appendix, k = hwc. In the Appendix C, there is only the Figure 10. How do you got the impression that k=1?

KellyWuuu commented 2 years ago

Thank you for your reply! Maybe I'm making a wrong connection. So, I'm looking at the third column of Figure 10, and the top and bottom heatmaps looked the same. Since the bottleneck is inserted at the same layer, conv4_1, I'm getting a sense that beta = 10 = 10/k.

Using k = hwc makes k an extremely large number; will this make beta too small?