monologg / GoEmotions-pytorch

Pytorch Implementation of GoEmotions 😍😢😱
Apache License 2.0
151 stars 46 forks source link

Tokenizer and Model loading for a fine-tuned model #2

Closed DevD1092 closed 4 years ago

DevD1092 commented 4 years ago

I am looking into loading the model and the tokenizer after it has been trained on a custom dataset. After training, I am able to produce pytorch_model.bin, config.json, tokenizer_config.json, special_tokens_map.json, training_args.bin, and vocab.txt for every checkpoint saved.

Is there any script where I can know how to load the saved checkpoints along with the tokenizer just like the example that you have provided here for your pre-trained model

tokenizer = BertTokenizer.from_pretrained("monologg/bert-base-cased-goemotions-ekman")
model = BertForMultiLabelClassification.from_pretrained("monologg/bert-base-cased-goemotions-ekman")

goemotions = MultiLabelPipeline(
    model=model,
    tokenizer=tokenizer,
    threshold=0.3
)

Thanks for the awesome repo!