Hi, I think your work is awesome and I really enjoy reading your paper. I am just curious about the computation of logical self-consistency,
When I browsing through the code for logical self-consistency, I notice that the mnli.pred file generated by the roberta-mnli model in the "roberta_mnli/self_logic/MNLI_SCORE/data/seq2seq" folder has 329 scores. But the dev_matched.tsv, which I suppose to contain the sentence pairs to feed into the model, contains 281 pairs. May I know if I miss anything here? I also don't understand what is the use of dev_mismatched.tsv?
Hi, I think your work is awesome and I really enjoy reading your paper. I am just curious about the computation of logical self-consistency,
When I browsing through the code for logical self-consistency, I notice that the mnli.pred file generated by the roberta-mnli model in the "roberta_mnli/self_logic/MNLI_SCORE/data/seq2seq" folder has 329 scores. But the dev_matched.tsv, which I suppose to contain the sentence pairs to feed into the model, contains 281 pairs. May I know if I miss anything here? I also don't understand what is the use of dev_mismatched.tsv?
Thanks in advance!