search

facebookresearch / clevr-dataset-gen

A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Other
577 stars 204 forks source link

Test Answers in CLEVR #2

Open Victoria2333 opened 6 years ago

Victoria2333 commented 6 years ago

I sent an email with my test file to @jcjohnson, i would like to know how long it will take for reply??? Thank for your attention~~~

jcjohnson commented 6 years ago

We followed up via email.

For others viewing this thread in the future, test-set answers should be formatted as a text file with one answer per line, in the same order as the questions in the JSON file; I will then evaluate using this script:

https://gist.github.com/jcjohnson/5796ff43e67835cb8e1de882a5867994

Victoria2333 commented 6 years ago

Thank you, i will modify my file format and send it to you later~~~

Victoria2333 commented 6 years ago

@jcjohnson Hi, do I need to include the other variables such as question_id to know which predicted answer is corresponding with the question?? However,it seems that it is enough to offer predicted answers in json file??And the answers are not sored in list,just need to be written in?? Thank you for attention~

jcjohnson commented 6 years ago

@Victoria2333 As mentioned above, your test-set answer should be a text file with one answer per line; you do not need to include question IDs. Please refer to the script I linked; you can make sure your file format is correct by using my script to evaluate your answers on the validation set.

Victoria2333 commented 6 years ago

I have already sent to you, Looking forward to your reply~

TimeLighter commented 6 years ago

Dear Justin. I have sent a request for evaluation of the test answer in CLEVR. Thanks in advance for your help.

jcjohnson commented 6 years ago

@TimeLighter I think I've responded to all pending test-set requests; please let me know if you still need test set results.

TimeLighter commented 6 years ago

I got your email. Thanks a lot :)

TimeLighter commented 6 years ago

Hi. I sent one more request with email. (modelC_test_answer.txt) I'm looking forward your replay. Thank you.

cyhbrilliant commented 5 years ago

@jcjohnson Hi Justin: I have already sent you a email with test_24_18.txt attachment file for evaluation about CLEVR 1.0, could you please replay as soon as possible. Thank you : ).

jcjohnson commented 5 years ago

@cyhbrilliant I responded via email

cyhbrilliant commented 5 years ago

@jcjohnson Thanks Justin: We‘ve seen that, and another two files send to you. Looking forward your reply, thanks very much.

yuzcccc commented 5 years ago

@jcjohnson Hi, johnson， I just send a mail to you to test CLEVR results. Hope to get a results soon. Thanks!

vmichals commented 5 years ago

@jcjohnson Sent an email requesting some test results. Hoping to get them soon. Thanks!

YL1113 commented 4 years ago

@jcjohnson Hi, johnson， I tried to test my latest VQA system in CLEVR_v1.0 dataset. I just send a mail to you to test CLEVR results. could you please reply me the results as soon as possible. very thanks! submit_answer.txt

ArrasL commented 4 years ago

Hi Justin, we sent you an email last week to your umich dot edu address with our test answers. We hope that's the right way to get the test accuracy ;-) Thanks in advance!

jcjohnson commented 4 years ago

@YL1113 @ArrasL replied over email

benbogin commented 4 years ago

We have just sent you our predictions as well and would appreciate your response, thanks so much!

cengzy14 commented 3 years ago

@jcjohnson Hi, johnson， I just send a mail to you to test CLEVR results (pred_mymodel_test.txt). Hope to get a result soon. Thanks a lot! (Sorry for send two emails via different email addresses, you can just reply one. Thanks in advance!

jcjohnson commented 3 years ago

@cengzy14 I replied on email. Good luck on CVPR!

cengzy14 commented 3 years ago

@cengzy14 I replied on email. Good luck on CVPR!

got it! thank you for your kindness! : )