How to load pretrained models from Hugging Face with this framework?

[filevich]

say i want to use/load a pre-trained model from hugging face, is something like this possible:

model = uf.BERTClassifier(from_pretrained='dccuchile/bert-base-spanish-wwm-cased')

thanks in advance, the idea of this framework is great!

[geyingli]

Really appreciate your question. Thank you

The thing is, hugging face implements BERT based on pytorch, but this framework is designed to meet the needs of tensorflow users. So we need to convert the pre-trained model from pytorch supported file into tensorflow supported checkpoint if we wish to use it. Since we haven't considered about this sort of issue in the past, UNIF does not support loading pre-trained model from pytorch checkpoint by now. We will try to implement it in the next a few days and let you know if we make it.

Before then, there is another way to meet this purpose: download the pre-trained model released by hugging face and run the conversion script convert_bert_pytorch_checkpoint_to_original_tf.py. Then run model = uf.BERTClassifier(..., init_checkpoint='./tf_checkpoint_dir') to load.

[filevich]

thank you very much for your kindness! 😊 And thank you for the framework, very handy. keep it going!

[geyingli]

Glad to hear that. I think I will. ^^