Are control codes required for finetuning?

salesforce / ctrl

Conditional Transformer Language Model for Controllable Generation

https://arxiv.org/abs/1909.05858

BSD 3-Clause "New" or "Revised" License

1.87k stars 208 forks source link

Are control codes required for finetuning? #75

Open umhau opened 4 years ago

umhau commented 4 years ago

I'm hoping to add some domain-specific knowledge to the Q&A control code, using text that is not in a Q&A format. Seems like that would be best accomplished by not adding my own control code to the text data that I finetune with. Am I going about this correctly?