unable to load the downloaded BERT model offline in local machine . could not find config.json and Error no file named ['pytorch_model.bin', 'tf_model.h5', 'model.ckpt.index'] |

[AjitAntony]

❓ Questions & Help

I have downloaded the bert model from the link in bert github page offline but unable to load the model offline .

from transformers import * model = BertForMaskedLM.from_pretrained("/Users/Downloads/uncased_L-12_H-768_A-12/")

Model name '/Users/Downloads/uncased_L-12_H-768_A-12/' was not found in model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-dbmdz-uncased). We assumed '/Users/Downloads/uncased_L-12_H-768_A-12/' was a path or url to a configuration file named config.json or a directory containing such a file but couldn't find any such file at this path or url.

below are the files present in /Users/Downloads/uncased_L-12_H-768_A-12/ bert_config.json bert_model.ckpt.data-00000-of-00001 bert_model.ckpt.index bert_model.ckpt.meta vocab.txt what should i do to load the downloaded model offline ?

since the error was saying config.json not found i changed the above 4 file names by removing the word bert from it.Below are the new file names

config.json model.ckpt.data-00000-of-00001 model.ckpt.index model.ckpt.meta vocab.txt now when i load the downloaded model offline i get a different error

from transformers import * model = BertForMaskedLM.from_pretrained("/Users/Downloads/uncased_L-12_H-768_A-12/")

Error no file named ['pytorch_model.bin', 'tf_model.h5', 'model.ckpt.index'] found in directory /Users/Downloads/uncased_L-12_H-768_A-12/ or from_tf set to False

python version:3.7 tensorflow version:1.12

[LysandreJik]

Hi, you're downloading one of the original implementation BERT models, which is in TensorFlow and you are trying to load it into one of our Pytorch models.

You can either download one of our checkpoints hosted on our S3 with:

from transformers import BertForMaskedLM

model = BertForMaskedLM.from_pretrained("bert-base-cased")

This model will now be available offline as it will be saved in your pytorch cache.

Or you can convert the BERT model you downloaded to a checkpoint readable by our library by using the script convert_bert_original_tf_checkpoint_to_pytorch.py

[AjitAntony]

@LysandreJik thank you . How to differentiate Bert tensor flow and pytorch models ?

I followed the instruction and create a PyTorch model using this pyhton code ->convert_bert_original_tf_checkpoint_to_pytorch.py

INFO:transformers.modeling_bert:Initialize PyTorch weight ['cls', 'seq_relationship', 'output_weights'] INFO:transformers.modeling_bert:Skipping cls/seq_relationship/output_weights/adam_m INFO:transformers.modeling_bert:Skipping cls/seq_relationship/output_weights/adam_v INFO:transformers.modeling_bert:Skipping global_step Save PyTorch model to /content/drive/My Drive/BMaskLang

the BMaskLang file was 402 MB size and it did not have any file extension ,now when i tired to load this pytorch model i get an error

_from transformers import BertForMaskedLM model = BertForMaskedLM.frompretrained("/content/drive/My Drive/BMaskLang") Error: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

Basically what im trying to do is train a BertForMaskedLM on a custom corpus .

what are the steps to train BertForMaskedLM model on custom corpus and After train model how to load it ? After loading how to test it on a new sentence ?

For example if there was a sentence in sample_text.txt corpus like "He went to space.He brought a moon"

if i want to test my pretrained BertForMaskedLM to check if it correctly predicts the masked word in sentences" He went to [Mask] .He brought a gallon [Mask]

so the model must predict the same words which was in sample_text.txt corpus "space","moon" rather than other words like "store","water" since it was trained on this sample_text.txt corpus .im expecting this behavior .Is this possible to pretrain and build language model using transformers bert ?

[hzmdschhc]

@LysandreJik thank you . How to differentiate Bert tensor flow and pytorch models ?

I followed the instruction and create a PyTorch model using this pyhton code ->convert_bert_original_tf_checkpoint_to_pytorch.py

INFO:transformers.modeling_bert:Initialize PyTorch weight ['cls', 'seq_relationship', 'output_weights'] INFO:transformers.modeling_bert:Skipping cls/seq_relationship/output_weights/adam_m INFO:transformers.modeling_bert:Skipping cls/seq_relationship/output_weights/adam_v INFO:transformers.modeling_bert:Skipping global_step Save PyTorch model to /content/drive/My Drive/BMaskLang

the BMaskLang file was 402 MB size and it did not have any file extension ,now when i tired to load this pytorch model i get an error

_from transformers import BertForMaskedLM model = BertForMaskedLM.frompretrained("/content/drive/My Drive/BMaskLang") Error: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

Basically what im trying to do is train a BertForMaskedLM on a custom corpus .

what are the steps to train BertForMaskedLM model on custom corpus and After train model how to load it ? After loading how to test it on a new sentence ?

For example if there was a sentence in sample_text.txt corpus like "He went to space.He brought a moon"

if i want to test my pretrained BertForMaskedLM to check if it correctly predicts the masked word in sentences" He went to [Mask] .He brought a gallon [Mask]

so the model must predict the same words which was in sample_text.txt corpus "space","moon" rather than other words like "store","water" since it was trained on this sample_text.txt corpus .im expecting this behavior .Is this possible to pretrain and build language model using transformers bert ?

I haved the same problem that how to load bert model yesterday. And now I found the solution.

1. run convert_bert_original_tf_checkpoint_to_pytorch.py to create pytorch_model.bin
2. rename bert_config.json to config.json

after that, the dictionary must have

config.json (BertForMaskedLM.from_pretrained() need it) pytorch_model.bin (BertForMaskedLM.from_pretrained() need it) vocab.txt (BertTokenizer.from_pretrained() need it)

python version 3.7 pytorch version 1.3.1 tensorflow version 2.0.0

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Deep1994]

Hi, you're downloading one of the original implementation BERT models, which is in TensorFlow and you are trying to load it into one of our Pytorch models.

You can either download one of our checkpoints hosted on our S3 with:

from transformers import BertForMaskedLM

model = BertForMaskedLM.from_pretrained("bert-base-cased")

This model will now be available offline as it will be saved in your pytorch cache.

Or you can convert the BERT model you downloaded to a checkpoint readable by our library by using the script convert_bert_original_tf_checkpoint_to_pytorch.py

hi, I can't open this link https://github.com/huggingface/transformers/blob/master/transformers/convert_bert_original_tf_checkpoint_to_pytorch.py, where can I find this script? Thanks!

[LysandreJik]

It's here

[Souravroych]

Traceback (most recent call last): File "/users/sroychou/BERT_text_summarisation/scripts/train_bert_summarizer.py", line 12, in from metrics import optimizer, loss_function, label_smoothing, get_loss_and_accuracy, tf_write_summary, monitor_run File "/users/sroychou/BERT_textsummarisation/scripts/metrics.py", line 16, in , , = b_score(["I'm Batman"], ["I'm Spiderman"], lang='en', model_type='bert-base-uncased') File "/users/sroychou/.local/lib/python3.7/site-packages/bert_score/score.py", line 105, in score tokenizer = AutoTokenizer.from_pretrained(model_type) File "/users/sroychou/.local/lib/python3.7/site-packages/transformers/tokenization_auto.py", line 298, in from_pretrained config = AutoConfig.from_pretrained(pretrained_model_name_or_path, kwargs) File "/users/sroychou/.local/lib/python3.7/site-packages/transformers/configuration_auto.py", line 330, in from_pretrained configdict, = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, kwargs) File "/users/sroychou/.local/lib/python3.7/site-packages/transformers/configuration_utils.py", line 382, in get_config_dict raise EnvironmentError(msg) OSError: Can't load config for 'bert-base-uncased'. Make sure that:

'bert-base-uncased' is a correct model identifier listed on 'https://huggingface.co/models'

or 'bert-base-uncased' is the correct path to a directory containing a config.json file