Closed loukasilias closed 3 years ago
I am trying to use LIME with a model consisting of BERT --> Pooling layer --> Dense (128 units) --> Dense (1 unit, sigmoid activation function). This is a binary classification code.
I have implemented the model as follows:
class MyModel(tf.keras.Model): def __init__(self, flag): super(MyModel,self).__init__() self.bert_model = TFBertModel.from_pretrained("bert-base-uncased") self.bert_model.trainable = flag self.layer_3 = Dense(units = 128, activation = 'relu') self.layer_4 = Dense(units = 1, activation = 'sigmoid') self.pooling = tf.keras.layers.GlobalMaxPool1D() def call(self,inputs): output_sentence_1 = self.bert_model(input_ids = inputs[0], attention_mask = inputs[1]) output_sentence_1 = self.pooling(output_sentence_1['last_hidden_state']) layer_output = self.layer_3(output_sentence_1) output = self.layer_4(layer_output) return output
I am trying to use LIME as follows:
from lime.lime_text import LimeTextExplainer explainer = LimeTextExplainer(class_names = ['truthful', 'deceptive']) tokenizer = BertTokenizer.from_pretrained("bert-base-uncased", do_lower_case=True) def predict_probab(STR): list_all = [] for strings in STR: z = tokenizer.encode_plus(strings, add_special_tokens = True, max_length = 512, truncation = True,padding = 'max_length', return_token_type_ids=True, return_attention_mask = True, return_tensors = 'np') inputs = [z['input_ids'], z['attention_mask']] k = [] k.append(float(model.predict(inputs).reshape(-1,1))) k.append(float(1-model.predict(inputs).reshape(-1,1))) k = np.array(k).reshape(1,-1) list_all.append(k) list_all = np.array(list_all).squeeze() return np.array(list_all) STR = [] for strings in X_test[:10]: STR.append(str(strings)) exp = explainer.explain_instance(str(STR[0]), predict_probab, num_features=100, num_samples=150)
I would like to ask you if the methodology of LIME is correct.
??? what is your problem
I am trying to use LIME with a model consisting of BERT --> Pooling layer --> Dense (128 units) --> Dense (1 unit, sigmoid activation function). This is a binary classification code.
I have implemented the model as follows:
I am trying to use LIME as follows:
I would like to ask you if the methodology of LIME is correct.