How to do finetuning with the pertrained model uploaded on hugging face?

[jasmoonli]

I love to use the pretrained Bioelectra model to do my own biomedical NER tasks, but currently I am still confused about how to fine-tune the pretrained model. Could you please provide any instruction or guide for me?

To learn the model structure, I decided to first run the bc2gm finetuning task and already downloaded the dataset from the github deposit you referenced in the publication. Then, I tried two ways to finetune the model, but either one failed:

1. I tried to finetune the model through codes you provided on Github https://github.com/kamalkraj/electra, but I'm not able to load the hugging face model (saved in bin file) with run_finetuning.py. (I could only successfully run the code with the model that the original ELECTRA team released.)
2. I tried to finetune the model through hugging face Transformers, but I'm not clear about how to load the dataset and design my own fine-tuning procedure with Transformers.

I wonder if it is possible to obtain the model checkpoints in a format that run_fintuning.py could correctly recognize, which might be similar to the model checkpoints the ELECTRA team provided? Or, can I transfer the hugging face checkpoints bin file to a usable format? I'll be really appreciated for all your help!

[kamalkraj]

Hi @jasmoonli,

You should be able to use this code to fine-tune the ELECTRA model with any dataset in the CoNLL-2003 format. https://github.com/huggingface/transformers/tree/master/examples/pytorch/token-classification

[jasmoonli]

Thanks for your guide! It helps a lot!