Can I used run_mlm.py in transformer for fine-tuning generator(mlm) of electra

google-research / electra

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

Apache License 2.0

2.31k stars 351 forks source link

Open ToanKGO opened 1 year ago

ToanKGO commented 1 year ago

Hi, I have a new corpus and want to fine-tune electra for better result. Can run file run_mlm.py makes my pre-trained electra works better