Closed ankushkundaliya closed 3 years ago
Hi @ankushkundaliya ,
thanks! Currently, PET does not support multi-label classification. You could try to rephrase the task as multiple single-label classifications. For example, let's say you want to know which topics from a given list (let's say, ["politics", "sports", "economics"]
) a text x
covers. You could reformulate this as three binary classification tasks, with PVPs like
P(x) = x. Is this text about politics? [MASK]
for the first taskP(x) = x. Is this text about sports? [MASK]
for the second taskP(x) = x. Is this text about economics? [MASK]
for the third taskand for all tasks, you could use a verbalizer that maps a match (i.e., the text is about the given topic) to Yes
(or True
) and a no-match to No
(or False
).
Hi @timoschick,
I am considering using the approach you explained above to turn a multi-label classification problem (with N labels) into N binary classification tasks. However, I wonder whether there's a big difference between using a single MLM to be fine-tuned rather than different MLMs for each pattern.
Because for example suppose that you have a very large amount of labels (such as 500), fine-tuning 500 MLMs is simply not feasible. Is using a single MLM appropriate?
Kind regards,
Niels
Hi @NielsRogge, we've run some experiments to answer that very question (I can't go into details as the corresponding paper is currently under review), and the short answer is: Yes, you can use a single MLM. However, you'll have to modify the source code of this library accordingly, as it does not support using a single MLM out-of-the-box.
Hi, Congratulations on this great work. I have a question about how to use PET for the multi-label classification task. Do I need to change the loss function defined for the current training?