search

timoschick / pet

This repository contains the code for "Exploiting Cloze Questions for Few-Shot Text Classification and Natural Language Inference"
https://arxiv.org/abs/2001.07676
Apache License 2.0
1.62k stars 283 forks source link

Is multi-token iPET be able to trained on a multi-class classification dataset? #82

Open GryffindorLi opened 2 years ago

GryffindorLi commented 2 years ago

Hi, I want to adapt muti-token iPET to a multi-classification task. However, all examples in the paper are binary classification. I am wondering can iPET be trained for multi-classification tasks?

chris-aeviator commented 2 years ago

I remember Timo Schick mentioning that this is not possible yet somewhere in this repository a while ago.

I used plain Bert to train my multi classification task

XiaoqingNLP commented 2 years ago

@GryffindorLi @chris-aeviator I think it is possible to support multiple categories. Initially, I understand that PET can be used on multiple tokens, so it should be able to be adapted to multi-category tasks. Besides PET, I can think of two ways to use it, but I'm not sure which one will work better

  1. classify multi-categories by number 1-N (N>10) and then predict the number to classify? This will lose semantic information
  2. splice the multiclassification labels after the text as a generation task and let the model do the generation of the labels?