Based on PET, which reads multiple patterned masked sentences and treats the output as a label, They improved it to be able to make multiple token predictions and surpasses GPT-3. GPT3 is a language model with billions of parameters, but it uses only about 1/1000th of that.
TL;DR
Based on PET, which reads multiple patterned masked sentences and treats the output as a label, They improved it to be able to make multiple token predictions and surpasses GPT-3. GPT3 is a language model with billions of parameters, but it uses only about 1/1000th of that.
Why it matters:
Paper URL
https://arxiv.org/abs/2009.07118
Submission Dates(yyyy/mm/dd)
2020/09/15
Authors and institutions
Timo Schick, Hinrich Schütze
Methods
Results
Comments