search

sairin1202 / Commonsense-Knowledge-Aware-Concept-Selection-For-Diverse-and-Informative-Visual-Storytelling

The implement of Commonsense Knowledge Aware Concept Selection For Diverse and Informative Visual Storytelling
MIT License
12 stars 2 forks source link

How to reproduce automatic metric in the paper. #5

Closed FeatherZhong closed 2 years ago

FeatherZhong commented 2 years ago

Hi, Thanks for sharing the amazing work, we observe that you do not seem to give the automatic evaluation score of BART model in the paper. And we calculated the score of stories in result/result.csv, which is lower than MCSM+RNN score in the paper. Could you share us the script to generate stories using MCSM+RNN model, or tell us what automatic evaluation score the BART model should have. Thanks a lot.

sairin1202 commented 2 years ago

stories in result/result.csv are from MCSM+Bart. I will upload the results from MCSM+RNN later. However, although MCSM+RNN can perform better on automatic scores, we think the generated stories are more informative with pretrained language model. (see human evaluation)

FeatherZhong commented 2 years ago

Got it, thx.