search

layer6ai-labs / xpool

https://layer6ai-labs.github.io/xpool/
110 stars 9 forks source link

why I cannot recurrence the result in the paper? #12

Closed Macro-1998 closed 2 years ago

Macro-1998 commented 2 years ago

Hello,I used the code to train on MSRVTT9k dataset, but I can't recurrence the result in the paper, in which R1 is 46.9, R5 is 72.8, R10 is 82.2 ,but the recurrence results R1 is 46.0, R5 is 71.4, R10 is 81.0. By the way, the lr in the github is 3e-5, but in your paper is 1e-5, after I use lr=1e-5 to train the model, the R1 promoted to 46.3 ,but is still far away from 46.9.

NoelVouitsis commented 2 years ago

There may be very slight performance differences caused by factors such as seed and compressing videos for training which affects which (12) frames are sampled. The MSR-VTT-9K results in our paper were obtained using the command in the README using lr=3e-5, default seed and compressed videos for training. Thanks!

Macro-1998 commented 2 years ago

thanks very much for your excellent work and your reply!

---Original--- From: "Noel @.> Date: Mon, Sep 12, 2022 01:17 AM To: @.>; Cc: @.**@.>; Subject: Re: [layer6ai-labs/xpool] why I cannot recurrence the result in thepaper? (Issue #12)

There may be very slight performance differences caused by factors such as seed and compressing videos for training which affects which (12) frames are sampled. The MSR-VTT-9K results in our paper were obtained using the command in the README using lr=3e-5, default seed and compressed videos for training. Thanks!

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

fake-warrior8 commented 2 years ago

Hello,I used the code to train on MSRVTT9k dataset, but I can't recurrence the result in the paper, in which R1 is 46.9, R5 is 72.8, R10 is 82.2 ,but the recurrence results R1 is 46.0, R5 is 71.4, R10 is 81.0. By the way, the lr in the github is 3e-5, but in your paper is 1e-5, after I use lr=1e-5 to train the model, the R1 promoted to 46.3 ,but is still far away from 46.9.

My recurrence results is R@1=46.1, R@5=73.3, using the given code, hyperparameters and the original video dataset downloaded from the given CLIP4CLIP github website.