Issues in visualizing a fine tuned model

[chikubee]

BertModel finetuned for a sequence classification task does not give expected results on visualisation. Ideally, the pretrained model should be loaded into BertForSequenceClassification, but that model does not return attentions scores for visualisation. When loaded into BertModel (0 to 11 layers), I assume the 11th layer (right before classification layer in BertForSequenceClassification) is the right layer to check attention distribution. But every word is equally attentive to every other word. I am wondering what can be the possible reasons and how I can fix it. Thanks.

[jessevig]

Hi, thanks for reporting this. As far as the issue of BertForSequenceClassification not returning attentions, where did you import BertForSequenceClassification from? If not imported from the special forked version of pytorch-transformers (in pytorch_transformers_attn directory) this can cause issues, so wanted to rule that out first.

[chikubee]

@jessevig I've used BertForSequenceClassification from the forked version only. I tried to initiate the BertModel with output_attentions=True as well, but something is still wrong. The attention scores for a token are scaled up for the fine tuned model in comparison to the pretrained bert uncased model with their scores being 0.06/0.07. The problem still persists. Thanks in advance for any leads.

[jessevig]

Good catch! There was indeed a bug and it has been fixed, so the tool now supports BertForSequenceClassification.

I believe I've loaded a pretrained model in the past in the following fashion:

model = BertForSequenceClassification(model_dir, num_classes=num_classes)

where model_dir contains config.json and pytorch_model.bin files. But this may not be appropriate in your case.

Please let me know if this works! Thanks again for catching this.

[chikubee]

@jessevig Sorry, doesn't work for me. What I am trying to do is basically visualize a BERT model fine tuned for sentiment analysis task.

If I load the model like model_state_dict = torch.load('models/bert_model_30_07_part3.pth') model = BertForSequenceClassification.from_pretrained('bert-base-uncased', state_dict=model_state_dict, output_attentions=True, num_classes=3)

The issue with this is that it doesn't return the attention scores. So visualization is not possible.

If I load the model like model = BertModel.from_pretrained('bert-base-uncased', state_dict=model_state_dict, output_attentions=True) The result is as reported.

Can you tell me how exactly can I achieve this?

[jessevig]

I'm still not able to recreate the issue for the latest version of the repo, unfortunately.

I created a colab notebook where I save a pre-trained model and the loaded the saved state_dict, and I am able to visualize the weights: https://colab.research.google.com/drive/1sgwcna3HhIbCqqIgYY3MxtoV7PWfpzF8.

Perhaps you could take a look and see if there is a difference with what you're doing? Otherwise, if you're willing to share your .pth file, I could try to debug what is going on.

Also, there could be an issue with the fine-tuned model that is causing it to give roughly equal attention weights. Do you see that in other layers as well? (besides layer 0 as in image?) Also, what do you see when you select "All" instead of "Sentence A -> Sentence A"?

[chikubee]

Thanks a lot @jessevig this works. I don't understand what lead to the issue though.

The changes I did after referring to your notebook was saved the model as an instance of BertForSequenceClassification loaded from pytorch transformers, while loaded it as an instance of the same class from the forked pytorch_transformers_attn.

Thanks a lot for your help.

[jessevig]

Glad it worked eventually. Will keep an eye out for similar issues to see if can get to root cause. Thanks again.