search

certainlyio / nordic_bert

Pre-trained Nordic models for BERT
Creative Commons Attribution 4.0 International
158 stars 11 forks source link

Reading the model into BERT? #22

Open klangborg opened 2 years ago

klangborg commented 2 years ago

I have an issue reading the model into BERT.

Can you share a piece of sample code on how to do that, or maybe have a look at what I am doing wrong?

This is my code:

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
config = BertConfig.from_json_file('dk/config.json')
model = BertForMaskedLM.from_pretrained(pretrained_model_name_or_path='dk/bert_model.ckpt.index', from_tf=True,config=config)

I get this error on the last line of the code above

 File "blanks.py", line 8, in <module>
    model = BertForMaskedLM.from_pretrained(pretrained_model_name_or_path='dk/bert_model.ckpt.index', from_tf=True,config=config)    
  File "\AppData\Local\Programs\Python\Python36\lib\site-packages\transformers\modeling_utils.py", line 1423, in from_pretrained
    model = cls.load_tf_weights(model, config, resolved_archive_file[:-6])  # Remove the '.index'
  File "AppData\Local\Programs\Python\Python36\lib\site-packages\transformers\models\bert\modeling_bert.py", line 156, 
in load_tf_weights_in_bert
    if pointer.shape != array.shape:
AttributeError: 'NoneType' object has no attribute 'shape'

Best regards Kristian