Based on PET, which reads multiple patterned masked sentences and treats the output as a label, They improved it to be able to make multiple token predictions and surpasses GPT-3. GPT3 is a language model with billions of parameters, but it uses only about 1/1000th of that.
TL;DR
Based on PET, which reads multiple patterned masked sentences and treats the output as a label, They improved it to be able to make multiple token predictions and surpasses GPT-3. GPT3 is a language model with billions of parameters, but it uses only about 1/1000th of that.![SuperGLUE Performance](https://user-images.githubusercontent.com/24524018/93409062-ff664580-f8d0-11ea-8b8d-330d62210997.png)
Why it matters:
Paper URL
https://arxiv.org/abs/2009.07118
Submission Dates(yyyy/mm/dd)
2020/09/15
Authors and institutions
Timo Schick, Hinrich Schütze
Methods
Results
Comments