search

neulab / guided_summarization

GSum: A General Framework for Guided Neural Abstractive Summarization
MIT License
113 stars 27 forks source link

Are we using oracle or automatic 'z' guidance signal at test time? #12

Closed DavidJiang9 closed 3 years ago

DavidJiang9 commented 3 years ago

https://github.com/neulab/guided_summarization/blob/f8cbfe49adf6f96e0ba6e08b0b890fd7f8420168/bert/highligted_sentence_data.py#L13

Hi, when I'm trying to reimplement the results on CNN/DM dataset, I find that the scripts above add the oracle guidance signals into processed CNN/DM data at both train and test time. However, according to the paper, we should use "pretrained extractive summarization models (BertExt or MatchSum) to perform automatic prediction at test time". Does that mean I should first use pretrained BertExt to generate the 'z' guidance signal for the test dataset? or i miss something. Thanks.

zdou0830 commented 3 years ago

Hi,

Yes, you should first use BertExt to generate the guidance signals for the test dataset. at test time we don't have access to the references so we need to use the automatic z. We also provide results when using the oracle z (Table 3) just to see the upper bound performance.

DavidJiang9 commented 3 years ago

I get it. Thanks a lot!

tiexueYL commented 3 years ago

I get it. Thanks a lot!

Could you tell me how to use BertExt to generate the guidance signals for the test dataset and how to add the guidance signals to BertAbs? I totally dont know what to do.Thank you very much!!

DavidJiang9 commented 2 years ago

Sry. I didn't notice this comment. Have you solved your problems? If not, I suggest you reading this paper and code in this repository. https://github.com/nlpyang/PreSumm