How to combine the explainer and the MTQ?

[windhxs]

Hello! I am intersted in your paper. But I am wondering how do you combine the explainer and MTQ or XMS to get a score. Can you tell me some details. Thank u.

[SinisterThaumaturge]

Hey sorry for the late response. What do you mean with how to combine the explainer and MTQ or XMS? If you mean how to get our best scores, you can look at our Jupyter Notebook explainability_captum.ipynb For all the Captum Explainers we used MonoTransQuest:

tr_model = MonoTransQuestModel(
    'xlmroberta',
    f'TransQuest/monotransquest-da-{SRC_LANG}_{TGT_LANG}-wiki', num_labels=1, use_cuda=torch.cuda.is_available()
)

We created a forward Functionwrapper or a Class Wrapper for MTQ depending on the Explainer.

def forward(inputs, attention_mask, labels,token_type_ids):
    prediction = tr_model.model(input_ids=inputs,attention_mask=attention_mask,token_type_ids=token_type_ids,labels=labels)
    return prediction[1]

class CaptumModel(torch.nn.Module):
    def __init__(self, model):
        super(CaptumModel, self).__init__()
        self.model = model

    def forward(self,inputs, attention_mask, labels,token_type_ids):
        return self.model(input_ids=inputs,attention_mask=attention_mask,token_type_ids=token_type_ids,labels=labels)[1]

captum_model = CaptumModel(tr_model.model)
explainer_integratedgradients = LayerIntegratedGradients(forward, tr_model.model.roberta.embeddings,device_ids=[0])
explainer_deeplift = LayerDeepLift(captum_model,tr_model.model.roberta.embeddings)
explainer_layerGradientXActivation = LayerGradientXActivation(captum_model,tr_model.model.roberta.embeddings,device_ids=[0])
explainer_occlusion = Occlusion(forward)

Now MTQ is wrapped into a Captum model and we can use the function explainer.attribute() to get our scores for the individual wordpieces. Like this in the case of explainer_deeplift:

input_dict = create_input(to_predict, tr_model)
  tr_model.model.eval()
  tr_model.model.zero_grad()
 attributions = explainer_deeplift.attribute(input_dict["input_ids"],additional_forward_args=(input_dict["attention_mask"],input_dict["labels"],input_dict["token_type_ids"]))
    attributions = summarize_attributions(attributions)

  attributions = attributions.cpu().detach().numpy()
  tokenizer = tr_model.tokenizer
  indices = input_dict["input_ids"][0].detach().tolist()
  all_tokens = tokenizer.convert_ids_to_tokens(indices)

The function create_inputs transforms the source and target sentence into inputs for MTQ.

to_predict = [src,tgt]
def create_input(to_predict, tr_model):
    dummy_label = 0
    if isinstance(to_predict[0], list):
        eval_examples = [
            InputExample(i, text[0], text[1], dummy_label) for i,text in enumerate(to_predict)]
    else:
        eval_examples = [InputExample(i, text, None, dummy_label) for i, text in enumerate(to_predict)]
    eval_dataset = tr_model.load_and_cache_examples(
                eval_examples, evaluate=True, multi_label=False,                        no_cache=True)

    eval_sampler = SequentialSampler(eval_dataset)
    eval_dataloader = DataLoader(eval_dataset, sampler=eval_sampler,      batch_size=tr_model.args.eval_batch_size)
    for batch in eval_dataloader:
        inputs = tr_model._get_inputs_dict(batch)
    return inputs

In the notebook Codeblock 11 you can see in function get_attr how we then combined the wordpiece scores to individual word scores.

I hope I could help you a little bit. If you have any more questions feel free to ask them.