search

microsoft / MASS

MASS: Masked Sequence to Sequence Pre-training for Language Generation
https://arxiv.org/pdf/1905.02450.pdf
Other
1.12k stars 206 forks source link

Summarisation: Has the model for download been pre-trained? #70

Closed shivgodhia closed 4 years ago

shivgodhia commented 4 years ago

Hi,

Thank you for the great work. I was wondering if the model you have for download (model-base-uncased) has already been pre-trained or do we need to do pre-training ourselves?

Thanks!

StillKeepTry commented 4 years ago

@hivestrung It has been pre-trained on Wikipedia + BookCorpus for 40 epochs. You can also do pre-training by your data if you have data in other domains.