search

Liyan06 / AggreFact

Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors (ACL 2023)
22 stars 1 forks source link

What are BertExt, BertExtAbs, and BertSum? #10

Closed forrestbao closed 8 months ago

forrestbao commented 8 months ago

Hi, I am looking at the model_name column of the data you shared. What are the three models BertExt, BertExtAbs, and BertSum? How are they related to BertSumExt and BertSumAbs of (Liu and Lapata, 2019)?

Liyan06 commented 8 months ago

BertExt, BertExtAbs come from PolyTope: https://github.com/hddbang/PolyTope/tree/master/outputs_with_human_annotation Those are indeed from Liu and Lapata, 2019. More details can be found in https://aclanthology.org/2020.emnlp-main.33.pdf.

forrestbao commented 8 months ago

Thanks. Then are there overlaps between BertExt/BertExtAbs and BertSum{Ext, Abs}? I do not wanna use the same or related samples twice.

Liyan06 commented 8 months ago

We have ensured that there is no duplicated data in the benchmark.

forrestbao commented 8 months ago

I am sorry that I have taken you so much time. You explained that BertExt and bertExtAbs come from Polytope. Then what about BertSum? Where does it come from?

Liyan06 commented 8 months ago

BertSum summaries are from FRANK.

forrestbao commented 8 months ago

Thanks for your reply.