search

atulkum / pointer_summarizer

pytorch implementation of "Get To The Point: Summarization with Pointer-Generator Networks"
Apache License 2.0
904 stars 243 forks source link

question about eval #52

Closed superxaiole closed 3 years ago

superxaiole commented 3 years ago

Hello, first of all, thank you for the code you wrote. After adding the coverage mechanism, after 500,000 iterations, the result of Rouges-1 rouges-2 Rouges-L was 0.5 lower than the original paper. May I ask why? And iteration 70000 was better than iteration 500000. 500000: rouge_1_fscore: 0.3890, rouge_2_f_score: 0.1705, rouge_l_fscore: 0.3579. 70000: rouge_1_fscore: 0.3912, rouge_2_f_score: 0.1705, rouge_l_fscore: 0.3567

atulkum commented 3 years ago

If you set the seed right it should be the same. I suspect this might come from some software version issue etc.