search

marcoancona / DeepExplain

A unified framework of perturbation and gradient-based attribution methods for Deep Neural Networks interpretability. DeepExplain also includes support for Shapley Values sampling. (ICLR 2018)
https://arxiv.org/abs/1711.06104
MIT License
720 stars 133 forks source link

Multiple Inputs with Embedding #47

Open diamondspark opened 4 years ago

diamondspark commented 4 years ago

I am trying to explain a CNN model. My model looks as following. Please advise what am I doing wrong. Thank you! ` 

with DeepExplain(session=sess) as de: 

    model = load_model(sent_model_paths)
    input_tensor = model.inputs
    embedding_tensor = model.layers[6].input
    pre_softmax =  model.layers[-2].output

    x_interpret = cnn.X_test

    embedding_out = sess.run(embedding_tensor, {input_tensor[0]: cnn.X_test})

    ys = [1, 0, 0]

    attributions = de.explain('grad*input', pre_softmax * ys, embedding_tensor, embedding_out)
`

This gives me the following error 

'
InvalidArgumentError: You must feed a value for placeholder tensor 'input_17_1' with dtype int32 and shape [?,200]
     [[{{node input_17_1}}]]
'

My model looks as following 

Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_16 (InputLayer)           (None, 200)          0                                            
__________________________________________________________________________________________________
input_17 (InputLayer)           (None, 200)          0                                            
__________________________________________________________________________________________________
input_18 (InputLayer)           (None, 200)          0                                            
__________________________________________________________________________________________________
input_19 (InputLayer)           (None, 200)          0                                            
__________________________________________________________________________________________________
input_20 (InputLayer)           (None, 200)          0                                            
__________________________________________________________________________________________________
embedding_4 (Embedding)         (None, 200, 300)     2519700     input_16[0][0]                   
                                                                 input_17[0][0]                   
                                                                 input_18[0][0]                   
                                                                 input_19[0][0]                   
                                                                 input_20[0][0]                   
__________________________________________________________________________________________________
activation_16 (Activation)      (None, 200, 300)     0           embedding_4[0][0]                
__________________________________________________________________________________________________
activation_17 (Activation)      (None, 200, 300)     0           embedding_4[1][0]                
__________________________________________________________________________________________________
activation_18 (Activation)      (None, 200, 300)     0           embedding_4[2][0]                
__________________________________________________________________________________________________
activation_19 (Activation)      (None, 200, 300)     0           embedding_4[3][0]                
__________________________________________________________________________________________________
activation_20 (Activation)      (None, 200, 300)     0           embedding_4[4][0]                
__________________________________________________________________________________________________
conv1d_16 (Conv1D)              (None, 200, 256)     77056       activation_16[0][0]              
__________________________________________________________________________________________________
conv1d_17 (Conv1D)              (None, 199, 256)     153856      activation_17[0][0]              
__________________________________________________________________________________________________
conv1d_18 (Conv1D)              (None, 198, 256)     230656      activation_18[0][0]              
__________________________________________________________________________________________________
conv1d_19 (Conv1D)              (None, 197, 256)     307456      activation_19[0][0]              
__________________________________________________________________________________________________
conv1d_20 (Conv1D)              (None, 196, 256)     384256      activation_20[0][0]              
__________________________________________________________________________________________________
max_pooling1d_16 (MaxPooling1D) (None, 1, 256)       0           conv1d_16[0][0]                  
__________________________________________________________________________________________________
max_pooling1d_17 (MaxPooling1D) (None, 1, 256)       0           conv1d_17[0][0]                  
__________________________________________________________________________________________________
max_pooling1d_18 (MaxPooling1D) (None, 1, 256)       0           conv1d_18[0][0]                  
__________________________________________________________________________________________________
max_pooling1d_19 (MaxPooling1D) (None, 1, 256)       0           conv1d_19[0][0]                  
__________________________________________________________________________________________________
max_pooling1d_20 (MaxPooling1D) (None, 1, 256)       0           conv1d_20[0][0]                  
__________________________________________________________________________________________________
concatenate_4 (Concatenate)     (None, 1, 1280)      0           max_pooling1d_16[0][0]           
                                                                 max_pooling1d_17[0][0]           
                                                                 max_pooling1d_18[0][0]           
                                                                 max_pooling1d_19[0][0]           
                                                                 max_pooling1d_20[0][0]           
__________________________________________________________________________________________________
dropout_4 (Dropout)             (None, 1, 1280)      0           concatenate_4[0][0]              
__________________________________________________________________________________________________
flatten_4 (Flatten)             (None, 1280)         0           dropout_4[0][0]                  
__________________________________________________________________________________________________
dense_4 (Dense)                 (None, 3)            3843        flatten_4[0][0]                  
==================================================================================================
Total params: 3,676,823
Trainable params: 1,157,123
Non-trainable params: 2,519,700
arundasan91 commented 3 years ago

Have you seen issue #11 ? I recently had this error in a custom network. I was working on Jupyter notebooks and clearing the Keras session prior to loading the model helped.

model_path = 'trained_models/name-of-the-model.h5'

K.clear_session()
model = load_model(model_path)

attrib_method = 'elrp'
#%% Loop begins
with DeepExplain(session = K.get_session()) as de:
    # for performance improvements: https://github.com/marcoancona/DeepExplain#performance-explainer-api
    input_tensor   = model.layers[0].input
    fModel         = Model(inputs = input_tensor, outputs = model.layers[-2].output)
    target_tensor  = fModel(input_tensor)
    explainer      = de.get_explainer(attrib_method, target_tensor, input_tensor)
    attributions = explainer.run(X, ys=pred)