search

ucinlp / autoprompt

AutoPrompt: Automatic Prompt Construction for Masked Language Models.
Apache License 2.0
594 stars 81 forks source link

Label Search of NLI #43

Closed XM-WANG closed 2 years ago

XM-WANG commented 2 years ago

Hi,

I noticed that the template is '[CLS] {sentence} [T] [T] [T] [P]. [SEP]' in the given command for NLI label search (See readme.md).

Is that correct? I suppose it should be '[CLS] {sentence_A} [P] [T] [T] [T] [T] {sentence_B}. [SEP]'.

And the keys of label_map should be in the capital, i.e entailment -> ENTAILMENT.

By the way, the repo is the clearest one I have ever seen! Many thanks for your share.

yasamanrazeghi7 commented 2 years ago

Hi, Thanks for pointing this out, It depends on the way that the dataset is preprocessed, the template we put in the paper which is {prem}[P][T]. . . [T]{hyp} is right. Keeping the sample template for label search and the prompt search will help the performance.

rloganiv commented 2 years ago

Thanks for catching this issue @XM-WANG as well as your kind words!

I've updated the example in the README to be consistent with the template and model used for trigger search.