How to load a different domain BERT-based pre-trained model?

[vr25]

I am trying to load the pre-trained model at pred/FinBERT-Pre2K_128MSL-500K FinBERT and trying to run the basic task of SST-2 (sentiment classification) using run_glue.py (https://huggingface.co/transformers/examples.html#glue).

But I run into the following error: OSError: Model name '/data/ftm/xgb_regr/FinBERT/pred/FinBERT-Pre2K_128MSL-250K' was not found in model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-dbmdz-uncased). We assumed '/data/ftm/xgb_regr/FinBERT/pred/FinBERT-Pre2K_128MSL-250K' was a path or url to a configuration file named config.json or a directory containing such a file but couldn't find any such file at this path or url.

Also, since this seems to be trained using TF, I was wondering if I can use PyTorch to load it.

Thanks.

[AdityaSoni19031997]

You can use torch to load it, convert the weights using the helper files; Not sure about your task, but for mine, i was using a BertModel with different pre-trained weights,

model = BertForSequenceClassification(MODEL_PATH, num_labels=len(np.unique(y_train_torch)))

(iirc from_tf is also a param to the function)

where MODEL_PATH is a directory that has

• config.json.
• your model [checkpoint/bin file].
• a vocab file as well.

[vr25]

Thank you for your reply.

The issue is a little different. All the 3 files: config.json, checkpoint, and vocab.txt are linked by a symbolic link in their repo. I am not sure how to get the actual files. Any suggestions for such a case?

[AdityaSoni19031997]

Well if you are running the experiments yourself, you will be downloading them either ways, just make changes where ever needed?

(i haven't tried passing a symbolic link to this func so not sure myself but it should work imo as well)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.