The effect of electra？

hsha0 / electra

ELECTRA MODEL NLP

13 stars 0 forks source link

The effect of electra？ #2

Open litaozijin opened 4 years ago

litaozijin commented 4 years ago

I wonder whether this codes have recurrented the effect the same as the paper described.And when do you plan to submit your pre-trained model ,thanks a lot.

hsha0 commented 4 years ago

I am testing it currently by pretraining Electra-small. I will update if I get some "good" results on GLUE.

hsha0 commented 4 years ago

Hi, I ran the pretraining of my model on the BookCorpus and Wikipedia dataset for a amount of steps and tested it on the GLUE dataset. The results show the model works. You can play around with it by following the guide in README. If you meet some issues, please don't hesitate to post a comment.