GPT vs BERT, under same computation and data resource, which one is better for downstream tasks like GLUE? - Githubissues

elyase / awesome-gpt3

4.59k stars 357 forks source link

GPT vs BERT, under same computation and data resource, which one is better for downstream tasks like GLUE? #16

Open guotong1988 opened 4 years ago

guotong1988 commented 4 years ago

Thank you very much.

EricLee8 commented 4 years ago

感觉BERT之类的用auto encoder模式来预训练的模型更适合GLUE吧，GPT比较擅长text generation