ALBEF VQA test_dev results failed in VQA official server

salesforce / ALBEF

Code for ALBEF: a new vision-language pre-training method

BSD 3-Clause "New" or "Revised" License

1.57k stars 199 forks source link

ALBEF VQA test_dev results failed in VQA official server #96

Closed sanyalsunny111 closed 2 years ago

sanyalsunny111 commented 2 years ago

I tried using the ALBEF VQA finetuned model along with the the vqa_test_dev.json (provided for downstream tasks) and ran evaluation on it. But when I uploaded the result.json file to the official server it failed (multiple times). Screenshot and error report attached as screenshots. Any feedback?

LiJunnan1992 commented 2 years ago

Hi, as written in the message. The VQA evaluation server has been updated and now needs ALL test questions even for evaluating on test-dev.