Pre-Trained Model (ipuneetrathore/bert-base-cased-finetuned-finBERT) loads in PyTorch but not Tensorflow

[turmeric-blend]

🐛 Bug

Information

Model I am using (Bert, XLNet ...): TFBertModel

Language I am using the model on (English, Chinese ...): English

The problem arises when using:

• [ ] the official example scripts: (give details below)
• [X] my own modified scripts: (give details below)

The tasks I am working on is:

• [ ] an official GLUE/SQUaD task: (give the name)
• [X] my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

This Works:

import torch
PRE_TRAINED_MODEL_NAME = 'ipuneetrathore/bert-base-cased-finetuned-finBERT'
model = BertForSequenceClassification.from_pretrained(PRE_TRAINED_MODEL_NAME)

# loads just fine

This Does NOT Work:

import tensorflow as tf
PRE_TRAINED_MODEL_NAME = 'ipuneetrathore/bert-base-cased-finetuned-finBERT'
model = TFBertForSequenceClassification.from_pretrained(PRE_TRAINED_MODEL_NAME)

# ERROR:

OSError: Can't load weights for 'ipuneetrathore/bert-base-cased-finetuned-finBERT'. Make sure that:

- 'ipuneetrathore/bert-base-cased-finetuned-finBERT' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'ipuneetrathore/bert-base-cased-finetuned-finBERT' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.

Expected behavior

It should load the model.

Environment info

• transformers version: 3.0.0
• Platform: Ubuntu 18.04
• Python version: 3.7
• PyTorch version (GPU?): 1.3.1 with GPU
• Tensorflow version (GPU?): 2.1.0 with GPU
• Using GPU in script?: Yes
• Using distributed or parallel set-up in script?: No

[LysandreJik]

Hello! That's because the user that uploaded that model didn't upload a TensorFlow version, only a PyTorch version. You can see it when you click on "show all files", you'll see that there is a pytorch_model.bin, but no tf_model.h5.

Here you can solve this by telling the TF model that you want to load from pytorch weights:

import tensorflow as tf
PRE_TRAINED_MODEL_NAME = 'ipuneetrathore/bert-base-cased-finetuned-finBERT'
model = TFBertForSequenceClassification.from_pretrained(PRE_TRAINED_MODEL_NAME, from_pt=True)  # <-- here

[julien-c]

and you could also ask the author (I believe @ipuneetrathore) if they could upload a TF version of the weights

[turmeric-blend]

hi @julien-c just wondering if there are any difference if the pytorch weights could be loaded through TF model anyway?

[julien-c]

Just that the PyTorch weights will have to be converted on the fly every time you instantiate your TF model