LRP for text. Visual Attribution for text based models.

[damzC]

Hi,

I am trying to understand how Deep Explain works with classification models in text. For the same, I have taken up a simple 5-Class sentence classifier, with the following network architecture:

# Define Model Parameters
numClasses = 5
maxWords = 20
embeddingDim = 200

# Define Network Parameters
sgdOptimizer = 'adam'
lossFun = 'categorical_crossentropy'
batchSize = 32
numEpochs = 10

# Define Embedding Layer
embedLayer = Embedding(name='embedding_layer',
                input_dim = nSymbols, output_dim = embeddingDim,
                input_length = maxWords, weights = [embedMatrix], mask_zero=True)

# Define Input Layer
seqInputLayer = Input(shape=(maxWords, ), dtype='int32', name='input_x')

# Define Network Architecture
embedSequences = embedLayer(seqInputLayer)
lstmLayer = LSTM(embeddingDim, name = 'lstm_layer')(embedSequences)
dropoutLayer = Dropout(0.5, name = 'dropout')(lstmLayer)
denseLayer = Dense(numClasses, name = 'dense')(dropoutLayer)
predictLayer = Activation("softmax", name = 'softmax')(denseLayer)
model = Model(inputs = seqInputLayer, outputs = predictLayer)

model.compile(loss=lossFun, optimizer=sgdOptimizer, metrics=["accuracy"])
model.fit(trainX, trainY, batch_size=batchSize, epochs=numEpochs, verbose=1)

I have not taken any validation set, as this is just for experimental purpose. I tried using Deep Explain context in the following way:

with DeepExplain(session=K.get_session()) as de:

    # Need to reconstruct the graph in DeepExplain context, using the same weights.
    # 1. Get the input tensor
    # 2. Get embedding tensor
    # 3. Target the output of the last dense layer (pre-softmax)

    inputTensor = model.get_layer("input_x").input
    embeddingTensor = model.get_layer("embedding_layer").output
    preSoftmax = model.get_layer("dense").output

    # Create a new model sharing the same layers until the pre-softmax layer
    fModel = Model(inputs=inputTensor, outputs = preSoftmax)
    targetTensor = fModel(inputTensor)

    # Sample Data for attribution
    sampleX = trainX[10:11]

    # Perform Embedding Lookup
    getEmbeddingOutput = K.function([inputTensor],[embeddingTensor])
    embeddingOutput = getEmbeddingOutput([sampleX])[0]

    ys = trainY[10:11]

    # Perform Attributions
    attributions = de.explain('elrp', preSoftmax * ys, embeddingTensor, embeddingOutput)

Unfortunately, I am running into an "InvalidArgumentError " error as follows: InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input_x_2' with dtype int32 and shape [?,20] [[Node: input_x_2 = Placeholder[dtype=DT_INT32, shape=[?,20], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Attached a file containing the error details: InvalidArgumentErrorDetails.txt

Can you please help me resolve this issue.

[marcoancona]

Hi! I think it is easier to create and train the model within the DeepExplain context from the beginning. Here is an example:

with DeepExplain(session=current_session) as de:  # <-- init DeepExplain context

    model = Sequential();
    model.add(Embedding(input_dim=4218+1, output_dim=32, input_length=100)); # input_length=29;, input_dim=max_words
    model.add(Flatten());
    model.add(Dense(100, activation='relu')); # input_shape=(max_words,)
    model.add(Dropout(0.5));
    #model.add(Dense(4, activation='softmax'));
    model.add(Dense(4, activation='linear'));
    model.add(Activation('softmax'));
    model.compile(loss='categorical_crossentropy',
                      optimizer='adam',
                      metrics=['accuracy']);
    model.summary();

    model.fit(X_train, y_train,
          batch_size=32,
          epochs=5,
          validation_data=(X_test, y_test),
          verbose=1,
          shuffle=True);

    # predict on test data
    y_pred = model.predict(np.array(X_test));
    y_test = np.array(y_test);

    # Evaluate the embedding tensor on the model input (in other words, perform the lookup)
    embedding_tensor = model.layers[0].output
    input_tensor = model.inputs[0]
    embedding_out = current_session.run(embedding_tensor, {input_tensor: X_test});

    xs = X_test;
    ys = y_test;
    # Run DeepExplain with the embedding as input
    attributions = de.explain('elrp', model.layers[-2].output * ys, model.layers[1].input, embedding_out);
    print("attributions shape --- {}".format(attributions.shape));

[damzC]

Hi,

Thanks for the response. My use case is more like testing unseen examples, after the model is built and correspondingly visualise the attributions on the input words. But is that possible based on your suggestion. I guess I will have to train the model each time, I want to test on new instances and check for attributions.

P.S. - The error I have mentioned above, is intermittent. I cannot figure out a way to fix it. There seems to be some intermittent issue with the input tensor.

Thanking you in advance. Any help would be really appreciated.

[damzC]

Hi Marco,

I tried your way. But I encountered the same error:

with DeepExplain(session=K.get_session()) as de:

    # Define Embedding Layer
    embedLayer = Embedding(name='embedding_layer',
                    input_dim = nSymbols, output_dim = embeddingDim,
                    input_length = maxWords, weights = [embedMatrix], mask_zero=True)

    # Define Input Layer
    seqInputLayer = Input(shape=(maxWords, ), dtype='int32', name='input_x')

    # Define Network Architecture
    embedSequences = embedLayer(seqInputLayer)
    lstmLayer = LSTM(embeddingDim, name = 'lstm_layer')(embedSequences)
    dropoutLayer = Dropout(0.5, name = 'dropout')(lstmLayer)
    denseLayer = Dense(numClasses, name = 'dense')(dropoutLayer)
    predictLayer = Activation("softmax", name = 'softmax')(denseLayer)
    model = Model(inputs = seqInputLayer, outputs = predictLayer)

    model.compile(loss=lossFun, optimizer=sgdOptimizer, metrics=["accuracy"])
    print model.summary()

    # Train the model
    model.fit(trainX, trainY, batch_size=batchSize, epochs=numEpochs, verbose=1)

    '''
    Need to reconstruct the graph in DeepExplain context, using the same weights.
    1. Get the input tensor
    2. Get embedding tensor
    3. Target the output of the last dense layer (pre-softmax)
    '''

    inputTensor = model.get_layer("input_x").input
    embeddingTensor = model.get_layer("embedding_layer").output
    preSoftmax = model.get_layer("dense").output

    # Create a new model sharing the same layers until the pre-softmax layer
    #fModel = Model(inputs=inputTensor, outputs = preSoftmax)
    #targetTensor = fModel(inputTensor)

    # Sample Data for attribution
    sampleX = trainX[10:11]

    # Perform Embedding Lookup
    getEmbeddingOutput = K.function([inputTensor],[embeddingTensor])
    embeddingOutput = getEmbeddingOutput([sampleX])[0]

    ys = trainY[10:11]

    relevances = de.explain('elrp', preSoftmax * ys, embeddingTensor, embeddingOutput)

The error is as follows:

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input_x_3' with dtype int32 and shape [?,20] [[Node: input_x_3 = Placeholder[dtype=DT_INT32, shape=[?,20], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

I am using keras version "2.2.2" and Tensorflow version "1.10.1" I have tried previous versions of Tensorflow (1.7.1), but the error is still there.

From what I can understand, I am passing value in the input tensor, but somehow the value is not getting populated.

[harkous]

@damzC. I'v been having similar issues. After debugging, it boiled down to the mask_zero=True. It seems that there is an issue when the model uses masking. A non-elegant fix for me was to train using masking, but to load a version of the model without it for attributions only. This is the function I used to convert my model before using it with DeepExplain:

def reload_model(model):
    model_json = model.to_json()
    model_json= model_json.replace('"mask_zero": true', '"mask_zero": false')
    new_model = model_from_json(model_json)
    new_model.set_weights(model.get_weights())
    return new_model

I didn't debug though whether the issue arises from a Keras bug or there is something to be adjusted in DeepExplain. These are related issues that were presumably fixed with previous versions of Keras:

• https://groups.google.com/forum/#!topic/keras-users/oOOrmBdeBoA
• https://github.com/keras-team/keras/issues/6169