search

timoschick / pet

This repository contains the code for "Exploiting Cloze Questions for Few-Shot Text Classification and Natural Language Inference"
https://arxiv.org/abs/2001.07676
Apache License 2.0
1.62k stars 282 forks source link

What is the logits? #57

Closed windhxs closed 2 years ago

windhxs commented 2 years ago

In class InputExample, it has a property logits, I am confused of this property, what does it used for ?

timoschick commented 2 years ago

Hi @windhxs, the final step of PET training is similar to knowledge distillation where the predictions of different models trained with different patterns are combined and a final model is trained on these combined predictions. The logits field in InputExample is used to store the combined (i.e., averaged) per-class logits to train the final model. This happens in line 254 - 261 of modeling.py.

Accordingly, the original InputExamples that you provide to the model should not have any logits.

windhxs commented 2 years ago

That's for answering!