[BUG] Problem with Add Layer in 1D ResNet network

[mg97tud]

I am using Innvestigate 2.0.0 and want to explain ECG classifications with guided backpropagation. My model has the form:

Pre

nb_classes = 3 n_feature_maps = 64 input_layer = keras.layers.Input(shape = (2700,1)) conv_1 = keras.layers.Conv1D(filters=n_feature_maps, kernel_size=16, padding='same')(input_layer) conv_1 = keras.layers.BatchNormalization()(conv_1) conv_1 = keras.layers.Activation('relu')(conv_1) conv_2 = keras.layers.Conv1D(filters=n_feature_maps, kernel_size=16, padding='same')(conv_1) conv_2 = keras.layers.BatchNormalization()(conv_2) conv_2 = keras.layers.Activation('relu')(conv_2) conv_2 = keras.layers.Dropout(.1)(conv_2) conv_3 = keras.layers.Conv1D(filters=n_feature_maps, kernel_size=16, padding='same')(conv_2) shortcut_1 = keras.layers.Add()([conv_3, conv_1])

Block 1

res_1 = keras.layers.Conv1D(filters=n_feature_maps, kernel_size=16, padding='same')(shortcut_1) res_1 = keras.layers.BatchNormalization()(res_1) res_1 = keras.layers.Activation('relu')(res_1) res_1 = keras.layers.Dropout(.1)(res_1) res_2 = keras.layers.Conv1D(filters=n_feature_maps, kernel_size=16, padding='same')(res_1) res_2 = keras.layers.BatchNormalization()(res_2) res_2 = keras.layers.Activation('relu')(res_2) res_2 = keras.layers.Dropout(.1)(res_2) res_3 = keras.layers.Conv1D(filters=n_feature_maps, kernel_size=16, padding='same')(res_2) shortcut_2 = keras.layers.Add()([res_3, res_1])

Block 2

res_4 = keras.layers.Conv1D(filters=n_feature_maps, kernel_size=16, padding='same')(shortcut_2) res_4 = keras.layers.BatchNormalization()(res_4) res_4 = keras.layers.Activation('relu')(res_4) res_4 = keras.layers.Dropout(.1)(res_4) res_5 = keras.layers.Conv1D(filters=n_feature_maps, kernel_size=16, padding='same')(res_4) res_5 = keras.layers.BatchNormalization()(res_5) res_5 = keras.layers.Activation('relu')(res_5) res_5 = keras.layers.Dropout(.1)(res_5) res_6 = keras.layers.Conv1D(filters=n_feature_maps, kernel_size=16, padding='same')(res_5) shortcut_3 = keras.layers.Add()([res_6, res_4])

Final

fin_1 = keras.layers.BatchNormalization()(shortcut_3) fin_1 = keras.layers.Activation('relu')(fin_1) fin_1 = keras.layers.GlobalAveragePooling1D()(fin_1) output_layer = keras.layers.Dense(nb_classes, activation='softmax')(fin_1)

And the problem arises when i want to execute: analysis_guidedback_resnet = analyzer_guidedback_resnet.analyze(X) in the "apply(layer: Layer, inputs: OptionalList[Tensor]) -> list[Tensor]:" function in the "init.py" in the "backend" folder. when it tries to execute "ret = layer(inputs[0])" in line 162.

The concrete error is:

Nachricht = 2 root error(s) found. (0) Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,2700,1] [[{{node input_1}}]] [[gradients_4/batch_normalization_6/moments/SquaredDifference_grad/scalar/_715]] (1) Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,2700,1] [[{{node input_1}}]] 0 successful operations. 0 derived errors ignored. Quelle = C:\LayerRelevanceDA\innvestigate\backend__init.py Stapelüberwachung: File "C:\LayerRelevanceDA\innvestigate\backend\init__.py", line 162, in apply ret = layer(inputs[0]) File "C:\LayerRelevanceDA\innvestigate\analyzer\gradient_based.py", line 223, in guided_backprop_reverse_relu_layer reversed_Ys = ibackend.apply(activation, reversed_Ys) File "C:\LayerRelevanceDA\innvestigate\backend\graph.py", line 1245, in reverse_model "stop_mapping_at_ids": local_stop_mapping_at_ids, File "C:\LayerRelevanceDA\innvestigate\analyzer\reverse_base.py", line 251, in _reverse_model return_all_reversed_tensors=return_all_reversed_tensors, File "C:\LayerRelevanceDA\innvestigate\analyzer\reverse_base.py", line 272, in _create_analysis return_all_reversed_tensors=return_all_reversed_tensors, File "C:\LayerRelevanceDA\innvestigate\analyzer\gradient_based.py", line 257, in _create_analysis return super()._create_analysis(*args, **kwargs) File "C:\LayerRelevanceDA\innvestigate\analyzer\network_base.py", line 166, in create_analyzer_model model, stop_analysis_at_tensors=stop_analysis_at_tensors File "C:\LayerRelevanceDA\innvestigate\analyzer\network_base.py", line 251, in analyze self.create_analyzer_model() File "C:\LayerRelevanceDA\visualize_importance.py", line 175, in (Current frame) analysis_guidedback_resnet = analyzer_guidedback_resnet.analyze(X)

When the error arises the inputs[0] Tensor from ret = layer(inputs[0]) has the Name gradients_7/AddN:0 so i guess there is some Problem with the Add Layer.

With my current setup I already investigated other models without problem, only now, when I use a ResNet structure with the Add layer i get the problem.

I also tried if I can explain my ResNet predictions with DeepTaylor and get the same error:

Nachricht = 2 root error(s) found. (0) Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,2700,1] [[{{node input_1}}]] [[gradients_6/batch_normalization_6/moments/SquaredDifference_grad/scalar/_759]] (1) Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,2700,1] [[{{node input_1}}]] 0 successful operations. 0 derived errors ignored. Quelle = C:\LayerRelevanceDA\innvestigate\analyzer\relevance_based\relevance_rule.py Stapelüberwachung: File "C:\Users\marcg\Desktop\DA_Code\LayerRelevanceDA\innvestigate\analyzer\relevance_based\relevance_rule.py", line 264, in tmp1 = [klayers.Multiply()([a, b]) for a, b in zip(X1, grads1)] File "C:\LayerRelevanceDA\innvestigate\analyzer\relevance_based\relevance_rule.py", line 264, in fn_tmp tmp1 = [klayers.Multiply()([a, b]) for a, b in zip(X1, grads1)] File "C:\LayerRelevanceDA\innvestigate\analyzer\relevance_based\relevance_rule.py", line 274, in apply self._layer_wo_act_positive, self._layer_wo_act_negative, Xs_pos, Xs_neg File "C:\LayerRelevanceDA\innvestigate\backend\graph.py", line 1245, in reverse_model "stop_mapping_at_ids": local_stop_mapping_at_ids, File "C:\LayerRelevanceDA\innvestigate\analyzer\reverse_base.py", line 251, in _reverse_model return_all_reversed_tensors=return_all_reversed_tensors, File "C:\LayerRelevanceDA\innvestigate\analyzer\reverse_base.py", line 272, in _create_analysis return_all_reversed_tensors=return_all_reversed_tensors, File "C:\LayerRelevanceDA\innvestigate\analyzer\deeptaylor.py", line 134, in _create_analysis return super()._create_analysis(*args, *kwargs) File "C:\LayerRelevanceDA\innvestigate\analyzer\deeptaylor.py", line 193, in _create_analysis return super()._create_analysis(args, **kwargs) File "C:\LayerRelevanceDA\innvestigate\analyzer\network_base.py", line 166, in create_analyzer_model model, stop_analysis_at_tensors=stop_analysis_at_tensors File "C:\LayerRelevanceDA\innvestigate\analyzer\network_base.py", line 251, in analyze self.create_analyzer_model() File "C:\LayerRelevanceDA\visualize_importance.py", line 175, in (Current frame) analysis_dt_resnet = analyzer_dt_resnet.analyze(X)

Maybe someone can guide me to setting an extra option i am not aware of.

[adrhill]

Hi @mg97tud,

this looks related to #292.

As a temporary workaround until we've fixed this issue, you could try to convert your BatchNormalization layers to Dense layers after training the model.