Can't reproduce the reported MSRVTT(zero-shot) results with the released model weights

TencentARC / MCQ

Official code for "Bridging Video-text Retrieval with Multiple Choice Questions", CVPR 2022 (Oral).

136 stars 16 forks source link

Can't reproduce the reported MSRVTT(zero-shot) results with the released model weights #11

Closed jiyt17 closed 2 years ago

jiyt17 commented 2 years ago

Hello, thank you for the code of MCQ! We utilize the released weights and follow the data settings, trying to reproduce MSRVTT ZS results. But our result(R@1) is about four points lower than the reported result in the paper. Is there any place we need to pay attention to? Thank you.

geyuying commented 2 years ago

The above error happens because of different library versions. The installation below can solve the problem.

conda create -n MCQ python=3.8
conda activate MCQ
pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
pip install -r requirements.txt

jiyt17 commented 2 years ago

ok thank you！

shufangxun commented 2 years ago

ok thank you！

Hello, do you reproduce the zero-shot result?