Open freedom6927 opened 3 years ago
@freedom6927 @sibeiyang this project, the author provided the Ref-Reasoning data set. 53% of the results were on the Ref-Reasoning data set. The author did not provide RefCOCO, RefCOCO+ and RefCOCOg Datasets, but provided a link with bottom-up-attention to process . Similarly, I also hope that the author can provide processed RefCOCO, RefCOCO+ and RefCOCOg Datasets, and their split
Hello, author, I found that your codes cannot reproduce the results in your paper. I can't reproduce refcoco's paper data with different pytorch versions and different parameter settings. Hope explanation. Thanks.
I couldn't reproduce the sgmn on the ref-reasoning dataset, too. The accuracy was 39.64% on the ref-reasoning test set after 10 epochs of training, while the paper says it produces 51.39% accuracy.
I use your released code to train dga model on the environment of python3 and torch 1.7. Then, i used the trained paratmeters to test your released dataset(test_expressions.json), the accuracy is 53.22%. However, the accuracy your write in papar is 86.64%, 84.79%, 78.31%, 80.21%, 80.26%(testA and testB of Refcoco and refcoco+ and test of recocog) , avg is about 79%. The number of test samples in your released test dataset is 34000+, and the number of all test samples of recoco,recoco+ and recocog is 5657+5059 +5726+4889+9602=30933(section 4 in paper dga). So i guess all test samples of recoco,recoco+ and recocog are included in your released test dataset. But the accuracy 53.22% is much lower than 79%. why is that. For the training, all parameters are the same as your proposed paper(Dynamic Graph Attention for Referring Expression Comprehension)