search

ronghanghu / snmn

Code release for Hu et al., Explainable Neural Computation via Stack Neural Module Networks. in ECCV, 2018
http://ronghanghu.com/snmn/
BSD 2-Clause "Simplified" License
72 stars 6 forks source link

How to validate VQA1 on CloudCV/EvalAI server? #5

Open monajalal opened 5 years ago

monajalal commented 5 years ago

We are not able to evaluate VQA1 on CloudCV/EvalAI server. They only have the evaluation code for VQA2.

I have attached the JSON file for VQA1 for you. Please let us know how you have reported evaluation numbers? https://drive.google.com/drive/folders/1Moq6qOimMpyCmT499KQgwoONGHaXVTgK?usp=sharing

We evaluated it here: http://evalai.cloudcv.org/

For VQA1:

screen shot 2018-11-07 at 3 41 42 pm screen shot 2018-11-07 at 3 42 25 pm screen shot 2018-11-07 at 3 42 45 pm

Doing the same procedure for VQA2 JSON file we get a ~90% accuracy.

mynameischaos commented 5 years ago

@monajalal https://competitions.codalab.org/competitions/6961#participate-submit_results