Was the model vit5-*-vietnews-summarization only trained with vietnews dataset?

[MinhDang685]

Hi VietAI team,

Thanks again for sharing your great work. I can easily reproduce the ROUGE scores result (as stated in your paper) on Vietnews dataset, with both VietAI/vit5-base-vietnews-summarization and VietAI/vit5-large-vietnews-summarization, by running the script provided here. But when I evaluate the 2 HuggingFace models, with Wikilingua (after change the tsv file to wikilingua test set to make it works for wikilingua) I found that the ROUGE scores are noticeably worse than the ones in the paper I got ROUGE-1 = 50.01, ROUGE-2 = 17.28, ROUGE-L = 29.92 for the large model and ROUGE-1 = 48.83, ROUGE-2 = 16.26, ROUGE-L = 29.37 for the base model.

And with the name of the models "vit5-*-vietnews-summarization" I guess the 2 summarization models were only trained with vietnews dataset. Could you help to correct or confirm my assumption?

Thanks. Minh

[justinphan3110]

@MinhDang685 our released huggingface model VietAI/vit5-base-vietnews-summarization and VietAI/vit5-large-vietnews-summarization are finetuned on vietnews dataset only. To reproduce the result on wikilingua you need to fine-tune it from the ViT5 checkpoints

[MinhDang685]

@justinphan3110 thank you for your reply