search

ok1zjf / VASNet

PyTorch implementation of the ACCV 2018-AIU2018 paper Video Summarization with Attention
MIT License
180 stars 48 forks source link

Can't reproduce results training #23

Open mallorbc opened 3 years ago

mallorbc commented 3 years ago

With the provided splits, I can not reproduce the results when training from scratch. I get an f1 scores of 61.33% and 47.79% for TVSum and SumMe respectfully. I am thinking that maybe the seed of 1234 is not the correct seed to be used for generating these results from scratch? Thanks!

ok1zjf commented 3 years ago

Hi Mallorbc, The published results were produces with the the github code as is, that is, including the seed. Later on we found out that the results slightly differed with later pytorch versions. It you setup you machine with the package versions documented in the readme.md you should be able to get the published values. Sorry I'm not able to help more. Best Jiri

mallorbc commented 3 years ago

No, thats plenty of help and makes perfect sense. Thanks

I do have a follow up though: How did you go about selecting the seed and the splits used for the final result? Using different seeds and or different splits leads to a noninsignificant change in the results. Thanks again!