Potential use with GPT-2?

timoschick / pet

This repository contains the code for "Exploiting Cloze Questions for Few-Shot Text Classification and Natural Language Inference"

https://arxiv.org/abs/2001.07676

Apache License 2.0

1.62k stars 283 forks source link

Potential use with GPT-2? #9

Closed fen0s closed 3 years ago

fen0s commented 3 years ago

This thing looks amazing so far! Anyone tried using it for minimized costs GPT training? If yes, can anyone share examples of generated text, and preferably, how to do it?

timoschick commented 3 years ago

Hi @fen0s, thanks for your kind words :) We are currently investigating if and how PET can be used in combination with generative models like GPT-2. We'll update this repo once our experiments are finished, but this is probably going to take a while.