idiap / fullgrad-saliency

Full-gradient saliency maps
Other
201 stars 31 forks source link

The saliency map given in your code does not satisfy the completeness condition. #6

Closed Heermosi closed 4 years ago

Heermosi commented 4 years ago

By what you designed to be a saliency map, there is 2 major problems:

  1. The code simply interpolate the bitmap, which in fact affected different areas of neurons, not just look like a zooming operation. Let's say, it's not a major problem, since the tracing of every single bias is utterly hard(I'm not very good at matrix).
  2. the gradients x biases and input gradient x input do compose the final score F(x). However, since the code interpolate the bitmap, the contribution of higher level duplicates many times, and were added up.

I think current implementation was helpful, but not accurate. The contribution value should be at least divided by the same zoom rate as it was interpolated. Then the sum up of the saliency map equals to the final score.

Any discussion is welcome.

suraj-srinivas commented 4 years ago

Hi,

I don't quite understand your first question, could you please re-phrase it?

Regarding the concern about completeness, it is true that the saliency map doesn't satisfy the completeness condition, and that is by design. It turns out no saliency map can satisfy two desirable conditions at the same time - completeness and sensitivity to local changes (i.e.; weak dependence on inputs). This is a disadvantage of all saliency maps which is shown in Section 3 of the paper.

I'm attaching the relevant portion in Section 4.2 of the paper, which talks about the issue.

image

Hope that helps.

suraj-srinivas commented 4 years ago

Hi, I'm closing the issue because of the lack of activity. Feel free to re-open if you have more questions.