Pretraining with checkpoints

dhlee347 / pytorchic-bert

Pytorch Implementation of Google BERT

Apache License 2.0

591 stars 179 forks source link

Closed abhi060698 closed 5 years ago

abhi060698 commented 5 years ago

Hey, if i want to restore a checkpointed model and perform pretraining, how would I do that?

dhlee347 commented 5 years ago

You can edit pretrain.py in order to use pretrain_file option

251: trainer.train(get_loss, model_file, None, data_parallel) --> 251: trainer.train(get_loss, model_file, pretrain_file, data_parallel)