search

timoschick / pet

This repository contains the code for "Exploiting Cloze Questions for Few-Shot Text Classification and Natural Language Inference"
https://arxiv.org/abs/2001.07676
Apache License 2.0
1.62k stars 285 forks source link

GenPET commands with GPT-2 #84

Closed yonwu closed 2 years ago

yonwu commented 2 years ago

Hi, @timoschick

Thanks for the excellent work on the PET project.

I am trying the GenPET now and successfully trigger training with the PEGASUS model. I saw in the code in wrapper, there is some GPT-2 related code.

If I would like to try GenPET with GPT-2, what would the command looks like, or is it enough that I only change the model to GPT2 instead of PEGASUS ?

Thanks for your answer in advance!

Best, Yongchao

timoschick commented 2 years ago

Hi @yonwu, using GenPET with GPT-2 currently is not supported: The GenPET code does some things that are specific to PEGASUS (or to encoder-decoder architectures in general). I currently don't have time to update the code myself to make it work with GPT-2, so if you want that, you probably have to make the necessary changes yourself 😞

yonwu commented 2 years ago

Hi, @timoschick Thanks for your answer! I close this issue then.