search

WillDreamer / Aurora

[NeurIPS2023] Parameter-efficient Tuning of Large-scale Multimodal Foundation Model
https://arxiv.org/abs/2305.08381
80 stars 7 forks source link

unable to reproduce your experimental #14

Open master-chou opened 9 months ago

master-chou commented 9 months ago

We are unable to reproduce your experimental results on the MSRVTT QA dataset, with an accuracy rate of around 31. May I ask for the reason, or can you provide some checkpoints

zhangbw17 commented 8 months ago

And I tried to reproduce the retrieval result using msrvtt, trained with command python3 -m torch.distributed.run --nproc_per_node=8 --master_port 12333 train_video_retrieval.py --config ./configs/retrieval_msrvtt.yaml --output_dir /tmp and tested with this command python3 -m torch.distributed.run --nproc_per_node=8 --master_port 12333 train_video_retrieval.py --config ./configs/retrieval_msrvtt.yaml --evaluate --pretrained path/to/ckpt --output_dir /tmp And the rank@1 is 45.0, which is much lower than the reported the result. may you provide the msrvtt checkpoints? Thansk.