TFXLMRoberta impossible to load base and large model with pretrained weight ?

[Shiro-LK]

🐛 Bug

Information

Model I am using (Bert, XLNet ...): XLMRoberta Language I am using the model on (English, Chinese ...):

The problem arises when using:

• [ x] the official example scripts: (give details below)

The tasks I am working on is: my own task or dataset: (give details below)

To reproduce

launch the command : model = TFXLMRobertaForSequenceClassification.from_pretrained(pretrained_model_name_or_path="xlm-roberta-large" ) Steps to reproduce the behavior:

1. from transformers import*
2. launch the command above
3. error :

TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

Expected behavior

model should be load.

Environment info

• transformers version: 2.6.0

• Platform: colab

• Tensorflow version (GPU?):2.1 gpu

• Using GPU in script?: yes

• Using distributed or parallel set-up in script?: no

[duyongan]

maybe u can use from_pt=True TFXLMRobertaForSequenceClassification.from_pretrained(pretrained_model_name_or_path="xlm-roberta-large", _frompt=True)

[Shiro-LK]

Hi, thanks for your answer. I got exactly the same message with "from_pt=True"

[Shiro-LK]

I am suspecting that the file does not exist. I succeed to load with TFXLMRoberta doing that :

m = XLMRobertaForSequenceClassification.from_pretrained("xlm-roberta-base", num_labels=1)
m.save_pretrained("./")
del m
model = TFXLMRobertaForSequenceClassification.from_pretrained("./", , num_labels=1, from_pt=True)

[LysandreJik]

Hello, indeed there is no official XLM-R checkpoint. You can use @jplu's checkpoint from the modelhub:

m = TFXLMRobertaForSequenceClassification.from_pretrained("jplu/tf-xlm-roberta-base", num_labels=1)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.