albermax / innvestigate

A toolbox to iNNvestigate neural networks' predictions!
Other
1.27k stars 233 forks source link

Question: pattern_based.py, how to backpropagate "a" value (patternNet) ? #98

Closed qinxie closed 6 years ago

qinxie commented 6 years ago

Hi, I am very appriciate to your code. Thank you so much.

I have two questions about PatternNet:

  1. what's the quantity that is backproparated? Is it 'a'?
  2. What's the backpropagation algorithm? Is it the same as Deep Taylor Decomposition?

Because I use deep Taylor decomposition as the backpropagate method in patternNet conputaion, the analysis result is very bad. but the patternAttribution result is very good. I don't know whether I used a wrong back propagate method. (I think after we get the patternNet "a" value, we have to back-propagate it to the "input" the same as Taloy decomposition). (My experiment is based on MNIST dataset.)

Thank you very much!

albermax commented 6 years ago

Hi qinxie,

  1. No, 'a' is used to backpropagate the value that is present at the last neuron.
  2. No, DeepTaylor and PatternNet are two different approaches. PatternAttribution uses the signals from PatternNet in the DeepTaylor framework. I think it's easiest if you look it up in the paper, in the appendix it is described in detail.

You should be careful as PatterNet and PatternAttribution/DeepTaylor are fulfilling two different tasks, the first is used to estimate the signal which is propagated to the output neuron, while the later is tries to identify the attribution of the input features to the final decision.

Hope this helps!

Cheers