search

yixinL7 / BRIO

ACL 2022: BRIO: Bringing Order to Abstractive Summarization
328 stars 43 forks source link

About the scoring mode #17

Open ZhouxiangFang opened 2 years ago

ZhouxiangFang commented 2 years ago

Hi, thank you for the good work! I am re-implementing BRIO and I am a little bit confused about the 'score mode' of the model. I mean, how does the scoring mechanism work, given several candidate summaries and corresponding source text? According to my understanding, the output of the model should be a sequence of token ids. Does 'the score of one summary' or 'the probability to generate one summary' mean the similarity between the output and candidate summary? Thank you for your patience!

yixinL7 commented 1 year ago

Hi, thank you for your interest in our project!

Does 'the score of one summary' or 'the probability to generate one summary' mean the similarity between the output and candidate summary?

You could think of this as the similarity between the source document and candidate summaries. And our model can be viewed as a reference-free evaluation model. Please see the introduction of our paper for a more detailed explanation.