richarddwang / electra_pytorch

Pretrain and finetune ELECTRA with fastai and huggingface. (Results of the paper replicated !)
324 stars 41 forks source link

how long to train electra base? (i have 30GB text, 1 GPU v100) #9

Closed BinhMinhs10 closed 3 years ago

richarddwang commented 3 years ago

It depends on how many iterations you want to train, if you use the same number of iteration as the paper (which is the default in my implementation), it takes 4 day on 16* TPU v3 s, according to the paper.