Use which type of data to calculate BLEU? Content plan or summary?

ratishsp / data2text-plan-py

Code for AAAI 2019 paper on Data-to-Text Generation with Content Selection and Planning

162 stars 46 forks source link

Use which type of data to calculate BLEU? Content plan or summary? #10

Closed LarryLee-BD closed 5 years ago

LarryLee-BD commented 5 years ago

Hi Ratish, Seems the code doesn't contain BLEU part. I have problems about how to calculate BLEU. I used the BLEU tools and inputed gold summaries and generate summaries, and got very high BLEU scores(about 40). I also tried content plan tuples files, and got high scores too(about 32).

LarryLee-BD commented 5 years ago

thanks

ratishsp commented 5 years ago

I used bleu perl script from https://github.com/moses-smt/mosesdecoder/blob/master/scripts/generic/multi-bleu.perl The model summary is used to compute BLEU. The command I used is ~/multi-bleu.perl ~/gold/val.summary < ~/gen/model-output-summary

LarryLee-BD commented 5 years ago

Nice