search

ZhangShiyue / QGforQA

MIT License
96 stars 21 forks source link

Script for computing Q-BLEU1 score? #2

Closed hugochan closed 4 years ago

hugochan commented 4 years ago

Hi,

Thank you for sharing this awesome code! I noticed that you also reported Q-BLEU1 score in your paper. I was wondering if you could kindly share the script for computing Q-BLEU1 score as well. I had some trouble using the official repo to compute Q-BLEU1, as detailed here.

Thank you!

hugochan commented 4 years ago

I computed the Q-BLEU1 using the following command: python answerability_score.py --data_type squad --ref_file references.txt --hyp_file hypotheses.txt --ner_weight 0.41 --qt_weight 0.2 --re_weight 0.36 --delta 0.66 --ngram_metric Bleu_1

Also, I was notified by the author of the official repo that there was previously a bug in the script which has been fixed. It works well now.