match neurons from CNN to pixels of input image

[zjjdcrn]

Thanks for sharing the awesome toolbox! I am just a beginner of artificial neural networks from cognitive neuroscience. I am struggling to find features(defined by CNN) that are critical for localizing the landmarks on faces. I noticed that the innvestigate and LRP toolbox could backprogate the activations of neurons from each layer to the raw image and this seems very relevant to my purpose: find the neurons in each layer that are critical for eyes/mouth localization. Could you give me some suggestion to achieve this? Many thanks!!!

[rachtibat]

Hi, thank you for your interest! I recommend using the new (unofficial) version on the branch updates_towards_tf2.0. In this version, you can select intermediate neurons. Initialize the explanations with arbitrary values and you have in general much more freedom. Please keep in mind, that there might be some bugs and we are constantly changing the code base. Also there is no real documentation.

Simple example:

Analyser = innvestigate.analyzer.LRPZ(model=model_wo_sm, neuron_selection=2, stop_mapping_at_layers=["dense"], r_init=1))

analysis = Analyser.analyze(X=x_test[:50])
analysis = next(iter(analysis.values()))

plt.figure("Heatmap")
plt.imshow(analysis[0].squeeze(-1), cmap="jet")
plt.show()

intermediate_explanations = Analyser.get_explanations()

Good luck!

[rachtibat]

due inactivity