Can not reproduce results on LSMDC dataset

albanie / collaborative-experts

Video embeddings for retrieval with natural language queries

https://www.robots.ox.ac.uk/~vgg/research/collaborative-experts/

Apache License 2.0

332 stars 55 forks source link

Can not reproduce results on LSMDC dataset #23

Closed ppj567 closed 3 years ago

ppj567 commented 3 years ago

Hi, I am trying to reproduce the results of CE on LSMDC dataset. I follow the instructions to evaluate the provided pre-trained model with config on LSMDC but get the results somehow lower than expected:

I evaluate it with the following script:

python test.py --config ./LSMDC_models/config.json --resume ./LSMDC_models/trained_model.pth --device 0 --eval_from_training_config

How could I reproduce the reported results below:

Many thanks!

ioanacroi commented 3 years ago

Hi,

Please be aware that all the numbers from the tables are reported over three seeded runs. The results you are obtaining are the exact numbers for the first seeded experiment. You can check the results of every seed in the log.

Cheers!