fix adapters with `*adapter_attention*`

[JoPfeiff]

🐛 Bug

Old versions of the adapters initialized *adapter_attention* which were never used but stored. I proposed a two stage fix:

• [ ] hot fix which does not log the warning that the parameters were not instantiated

• [ ] remove the parameters from all adapters

Information

Model I am using (Bert, XLNet ...): e.g. RoBERTa-Base

Language I am using the model on (English, Chinese ...): English

Adapter setup I am using (if any): many but e.g.

model = AutoModel.from_pretrained("roberta-base")
model.load_adapter("comsense/csqa@ukp", "text_task", config="{'using': 'pfeiffer'}")

The problem arises when using:

• [ x] the official example scripts: (give details below)
• [ ] my own modified scripts: (give details below)

The tasks I am working on is:

• [x ] an official GLUE/SQUaD task: (give the name)
• [ ] my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

1. load the adapter, you will get a warning for parameters which are not required

Expected behavior

no warning

Environment info

• transformers version:
• Platform:
• Python version:
• PyTorch version (GPU?):
• Tensorflow version (GPU?):
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

[calpt]

hot fix which does not log the warning that the parameters were not instantiated

can we keep the logging and reduce the log level from "warn" to "info"?

[arueckle]

Can we just remove the weights from the checkpoints?

import sys
from shutil import copyfile

import torch

chckpt_path = sys.argv[1]
copyfile(chckpt_path, chckpt_path + '.backup')
chckpt = torch.load(chckpt_path, map_location=torch.device('cpu'))
chckpt_new = dict()

for k, w in chckpt.items():
    if 'adapter_attention' not in k:
        chckpt_new[k] = w
    else:
        print('unwanted key: {}'.format(k))

torch.save(chckpt_new, chckpt_path)