moboehle / B-cos

B-cos Networks: Alignment is All we Need for Interpretability
Other
103 stars 11 forks source link

Can you help with some of the terms in the paper #4

Closed scutfrank closed 1 year ago

scutfrank commented 1 year ago

Hi , thanks for your work. It really open my mind for the interpretable of DNN with a replacement of linenar transform . But when reading the paper ,i have some point not understanding well as below, so if you can help to answear it ? 1.What kind of metric is the 'localisation metric',what 's the physical meaning? (Fig 6) 2.What's the meaning of baseline 'orcale attribution' and 'uniform attribution' ?(Fig 6) image 3.Is there some parameter refer to the area in red ? Or just for better visual ? (Fig 2) image

moboehle commented 1 year ago

Hi,

regarding 1+2:

the localisation metric is described in detail in CoDA Networks, CVPR 2021, see eq. (11) in that paper. In short, the localisation score measures the fraction of positive attribution scores in the correct grid location over all positive attributions for a given grid image (for an example of such a 'grid image', see Fig. 3 in the B-cos Networks paper).

As such, the best possible score that can be achieved in this metric is 1, i.e., positive attributions only fall into the correct grid image. In Fig. 6, this is denoted as the "Oracle attributions" score. Similarly, if an attribution method were to randomly place the positive attributions over the grid, the expected localisation score would be $1/n$ with $n$ the number of images in the grid; this, in turn, is denoted as the "Uniform attributions" score in Fig. 6.

regarding 3:

Thank you for pointing this out, it seems like the meaning of the red shading got lost in some revision of the paper. The strength of the red colour denotes the strength of positive activations of the B-cos transform for any position. E.g., as can be seen in the last row, if B is large the B-cos transform only activates for samples that are placed along the direction in which the weight vector points. In contrast, for B=1, all points that are at the same distance from the decision boundary yield the same output.

I hope this clarifies your questions!

scutfrank commented 1 year ago

I am clear now,thank you!