search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
132.76k stars 26.45k forks source link

Load saved Pytorch model into Tensorflow or convert from Pytorch model to TF #9204

Closed Jess0-0 closed 3 years ago

Jess0-0 commented 3 years ago

Hi,

Thanks for this awesome framework! I have trained and saved an XLMRoberta model in PyTorch and I'm wondering if there is any way I can load the model into TFRobertaForSequenceClassification class. Or if there are ways to convert the checkpoint to TensorFlow checkpoints so that it can be load by TF2.

I came across this file https://github.com/huggingface/transformers/blob/master/src/transformers/convert_pytorch_checkpoint_to_tf2.py and I'm wondering if there are any docs or instructions to use the script.

Thanks!

patrickvonplaten commented 3 years ago

Hey @Jess0-0,

You can load TF checkpoints into PT and vice-versa via:

XLMRoberta.from_pretrained(...., from_tf=True)

or

TFXLMRoberta.from_pretrained(...., from_pt=True).

Since XLMRobertaForSequenceClassification is just an alias of RobertaForSequenceClassification there should be no problem in doing 

TFRobertaForSequenceClassification.from_pretrained("<your/path/to/saved/xlm/roberta/pytorch/dir>", from_pt=True)