timoschick / pet

This repository contains the code for "Exploiting Cloze Questions for Few-Shot Text Classification and Natural Language Inference"
https://arxiv.org/abs/2001.07676
Apache License 2.0
1.62k stars 283 forks source link

PET's final classifier for RAFT benchmark B77 task #71

Open LiweiPeng opened 2 years ago

LiweiPeng commented 2 years ago

Hi, Thanks for publishing PET's excellent results on RAFT benchmark in your recent "True Few-Shot Learning with Prompts – A Real-World Perspective" paper. The best practices in the paper are very useful for real-world examples.

In the paper section 5 "PET for Real-World Tasks" -> "Handling Many Labels", you mentioned ".... We thus train the final model as a regular classifier with 77 different output classes; for training this classifier on an input x, we set the target probability of each output y proportional to the probability of True being the correct output for (x, y) according to our ensemble of binary classifiers.".

Can you provide more details on how the above is implemented in PET framework? Can you share your code on this? Thanks,

Liwei