Faithfulness and factuality annotations of XSum summaries from our paper "On Faithfulness and Factuality in Abstractive Summarization" (https://www.aclweb.org/anthology/2020.acl-main.173.pdf).
I am calculating Spearman's correlation coefficient (scipy.stats.spearmanr) using data in eval_scores_xsum_summaries.csv
My calculated results (See attached figure) on R1, R2, RL, BERTScore are the same as the results reported in Table 4 in your paper, but the results on Entailment are much lower than your results. (0.25 vs 0.431; 0.19 vs 0.264).
Would you please let me know any special operations you did on entailment scores to get results in your paper?
Hi,
I am calculating Spearman's correlation coefficient (scipy.stats.spearmanr) using data in eval_scores_xsum_summaries.csv
My calculated results (See attached figure) on R1, R2, RL, BERTScore are the same as the results reported in Table 4 in your paper, but the results on Entailment are much lower than your results. (0.25 vs 0.431; 0.19 vs 0.264).
Would you please let me know any special operations you did on entailment scores to get results in your paper?