Langboat / Mengzi

Mengzi Pretrained Models
Apache License 2.0
534 stars 63 forks source link

Input prefix of the model mengzi-t5-base #46

Closed KarenMars closed 2 years ago

KarenMars commented 2 years ago

Hi,

I have a question regarding the input of the model mengzi-t5-base. In the original paper of T5, it mentions that "we need to add the task-specific prefix to the original input sequence before feeding it to the model". I wonder that if I want to perform text summarize task with mengzi-t5-base or other downstream tasks, do I need to add some prefix, and what the prefix should be. Thank you very much for your help, looking forward to your reply.

Ag2S1 commented 2 years ago

mengzi-t5-base was trained following google's t5 v1.1. It was trained on general corpus only without mixing downstream tasks. You can directly finetune it on your summarisation dataset.

KarenMars commented 2 years ago

Thank you very much, your answer is very helpful.