search

microsoft / MASS

MASS: Masked Sequence to Sequence Pre-training for Language Generation
https://arxiv.org/pdf/1905.02450.pdf
Other
1.11k stars 206 forks source link

How to use MASS for Style Transfer? #143

Closed him-mah10 closed 4 years ago

him-mah10 commented 4 years ago

Hello, I want to finetune for style transfer task.

I have following doubts:

  1. If I want to pre-train, what should be the value of arguments --lgs and --mass_steps? As per my understanding both of them should have value en.
  2. When I- finetune what should be the value of argument --bt_steps? Now for en to fr conversion, the value was en-fr-en, but for style transfer how to finetune?
  3. Can I use MASS-base-uncased as already pre-trained model and use it for fine tuning?

Thank-you!

StillKeepTry commented 4 years ago
  1. Maybe you can consider A-style corpus as en and B-style corpus as fr. and training.
  2. If you already have parallel data, maybe use --mt_steps 'A-B'. You can also use --bt_steps 'A-B-A' and 'B-A-B' for on-the-fly back-translation.
  3. Of course, you can use Mass-base-uncased for directly fine-tuning.
him-mah10 commented 4 years ago

@StillKeepTry could you provide me a way to fine tune mass-base-uncased for style transfer tasks? TIA