Open EoinKenny opened 5 years ago
Yes, it possible to use an intermediate hidden layer as input to generate attributions (indeed, this is necessary for layers that stop gradient propagation, ie. embedding lookups)
You need to make sure the sample
parameter when you call explain(method_name, target_tensor, input_tensor, samples, ...args)
is the numeric input at that hidden layer (one that can be used to feed input_tensor
).
For example, when you have an embedding lookup you need to run first your model up to the interest hidden layer, take the numerical result at that point, and use it when calling explain
.
If that is not clear enough, it would be helpful to see a snippet of your code to check where the problem is.
Thanks for the reply!
So this is the normal code which works fine
with DeepExplain(session=K.get_session()) as de: # <-- init DeepExplain context
input_tensor = final_model.input
fModel = Model(inputs=input_tensor, outputs=final_model.layers[-2].output)
target_tensor = fModel(input_tensor)
xs = image_matrix
ys = np.zeros((1000))
ys[900] = 1
x = de.explain('elrp', target_tensor * ys, input_tensor, xs)
And this is how I modify it to get the contributions at the conv layer at layer [-7]
with DeepExplain(session=K.get_session()) as de: # <-- init DeepExplain context
input_tensor = final_model.layers[-7].input
fModel = Model(inputs=input_tensor, outputs=final_model.layers[-2].output)
target_tensor = fModel(input_tensor)
xs = conv_act
ys = np.zeros((1000))
ys[900] = 1
x = de.explain('elrp', target_tensor * ys, input_tensor, xs)
The error message is
ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_1:0", shape=(?, 224, 224, 3), dtype=float32) at layer "input_1". The following previous layers were accessed without issue: []
For some background here, I'm using the pre-trained VGG16 model. I modified the last two layers to be a separate dense and activation layer (softmax) with the usual 1000 classes.
The variable conv_act
is a matrix of shape (1, 14, 14, 512) - Which does match the conv layer input.
I don't see an obvious problem with your code. Seems you have some disconnected operations on the graph but that should not happen with within that snipped of code. Could you provide a minimal working example that I can run? Also, is the problem still there if you use grad*input
as method?
Thanks a lot for the help again. Here's a fully coded example with the error. The same error occurs with grad*input
also. Is this because you need to separate out the activation part of all layers in the network maybe?
import keras.backend as K
import numpy as np
from deepexplain.tensorflow import DeepExplain
from keras.layers import Dense, Activation
from keras.applications.vgg16 import VGG16
from keras.models import Model, Sequential
# Load VGG16 model and change last layer to be split with dense and Activation
vgg = VGG16(include_top=True, weights='imagenet')
vgg.save_weights('imagenet.h5')
x = Dense(1000)(vgg.layers[-2].output)
x = Activation('softmax')(x)
vgg_new = Model(vgg.input, x)
vgg_new.load_weights('imagenet.h5')
# Just use an array of 0's for this example
image_instance = np.zeros((1,224,224,3))
# Get activations of layer [-7]
conv_act_model = Model(inputs=vgg_new.input, outputs=vgg_new.layers[-7].output)
conv_act = conv_act_model.predict(image_instance)
# Get contributions of convolutional layer [-7]
with DeepExplain(session=K.get_session()) as de: # <-- init DeepExplain context
input_tensor = vgg_new.layers[-7].input
fModel = Model(inputs=input_tensor, outputs = vgg_new.layers[-2].output)
target_tensor = fModel(input_tensor)
xs = conv_act
ys = np.zeros((1000))
ys[900] = 1
answer = de.explain('elrp', target_tensor * ys, input_tensor, xs)
I'm just going to bump this in case! Thanks again for the help.
Is it possible to use DeepExplain to get contribution scores for a layer in the middle of the network?
I presume you have to change this line
target_tensor = fModel(input_tensor)
But if I replace input_tensor with a conv layer like model.layers[-7].input (for example) I get the error message.ValueError: number of input channels does not match corresponding dimension of filter
Is it possible to do this?