Closed gyjung975 closed 6 months ago
Hi,
Thanks for pointing out! We have uploaded the wrong log. I have uploaded the correct one. Would you please help check if the number is correct this time? Thanks!
It works well! Thank you
Can I also get logs for VQAv2 dataset?
Sorry that we are using the VQAv2 test-dev set, it does not come with gold answers (it is evaluated on Eval.ai) so we cannot share that in Github. I can send it via email if you need it. Would you please drop me an email so that I know where to send it?
gyjung975@uos.ac.kr Thank you for sending it here.
Just sent!
Hi, thanks for your interesting work.
I wonder how you evaluated your final results of OK-VQA.
Paper says it was evaluated with soft accuracy of VQAv2, so I tried to evaluate your OKVQA_val_gpt3.json in Evaluation Logs using VQA evalutation code. https://github.com/GT-Vision-Lab/VQA
It shows 58.89 score, but 60.4 in paper. I used "mscoco_val2014_annotations.json" file in OK-VQA website for annotation file.
Didn't you use the VQA evaluation code? or the log files are not final result?
Thank you