search

yixinL7 / SimCLS

Code for our paper "SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization", ACL 2021
183 stars 25 forks source link

A question in Table 1: Results on CNNDM. #1

Closed Doragd closed 3 years ago

Doragd commented 3 years ago

Thanks for your insightful work. However, I am confused by some details in the Table 1. Is the model which derives the 'Max' result also trained by contrastive learning ? or simply sampled from different beam search process?

yixinL7 commented 3 years ago

Thanks for your interest in our work! In Table 1, the max oracle is built by selecting the best candidate (highest ROUGE score) from the generated summaries. Therefore, it is not trained by contrastive learning.

Doragd commented 3 years ago

I would like to ask that the best performance can be obtained by choosing the highest rouge among all candidates, why do we need the second stage of contrastive learning?

yixinL7 commented 3 years ago

As ROUGE scores are computed between the reference summaries and generated summaries, the Max Oracle is not really a model itself (since it has access to the reference). A model, like the evaluation model in our work, needs to select the candidates based on the candidates and the source documents.

Doragd commented 3 years ago

Thanks a lllllot！！！ I think i have been get the "reference-free" idea！