search

RUCAIBox / TextBox

TextBox 2.0 is a text generation library with pre-trained language models
https://github.com/RUCAIBox/TextBox
MIT License
1.08k stars 117 forks source link

How to use the other pre-training objectives like masked sequence-to-sequence modeling? #336

Closed zhl98 closed 1 year ago

zhl98 commented 1 year ago

Hello, I am very interested in your work. I know ”--pretrain_task=denoising“ can use denoising auto-encoding. However, I did not see the relevant parameter descriptions and restrictions in the code. When I want to use other pre training methods, How do I set this parameter?

Thank you very much for your work, and I hope you can tell me the method when you have time.

StevenTang1998 commented 1 year ago

We are now testing other pre-training objectives, and we will release it soon. Thanks for your reporting!