search

sibeiyang / sgmn

Graph-Structured Referring Expressions Reasoning in The Wild, In CVPR 2020, Oral.
MIT License
116 stars 15 forks source link

Higher validation accuracy #6

Open chengyang317 opened 4 years ago

chengyang317 commented 4 years ago

Hi, Thank you for your great work. When I run the bash experiments 'bash experiments/scripts/train_cmrin.sh 2' and 'bash experiments/scripts/train_dga.sh 1', they show higher validation accuracy compared to your paper. They are 'Epoch: [9] Loss 0.0899 Prec 0.5305' and 'Epoch: [9] Loss 0.0847 Prec 0.5283', while your paper shows they are 45.43 and 45.37. I just change the batch size to 48 and the lr as 0.75e-4. Have you encountered this? By the way, I want to test my model in the test split, How can I do such test?

Than you!

sibeiyang commented 4 years ago

Hi @chengyang317 ,

Are you using the object features provided by this repository? If yes, the validation accuracy for them is not 45.43 and 45.37 (with Faster-RCNN features), respectively. We provide the bottom-up features here, and the accuracy for models with bottom-up features is given in Table 1 with * (https://sibeiyang.github.io/publication/sgmn/sgmn_cvpr.pdf). BTW, I did not optimize hyperparameters for all the models.

I will upload the test split to the shared google folder.

Best, Sibei

ZhangWenjing1 commented 4 years ago

Hello, when downloading the dataset, the uploader did not give permission. Can you share the dataset you downloaded! Thank you！

ZhangWenjing1 commented 4 years ago

Hi, Thank you for your great work. When I run the bash experiments 'bash experiments/scripts/train_cmrin.sh ' and 'bash experiments/scripts/train_dga.sh ', I want to test my model in the test split, do you upload the test split to the shared google folder. Than you!