Closed Artanic30 closed 10 months ago
From the illustration in paper Figure4, I find these questions are from OKVQA validation set.
Thank you for your interest in our work, and sorry for the late reply! For in-context learning, we use OK-VQA train set as in-context examples for the evaluation on OK-VQA dev set. That is the common practice for recent few shot learning works and it gives better score than examples from other dataset.
Thanks for your reply. I thinks this setting is fair enough for me.
Hi, thanks for your nice work. I'm wondering what is the source of VQA samples used for in-context learning during OKVQA evaluation. Is it the VQAV2 training set, OKVQA training set or OKVQA validation set? I would appreciate it if you could provide information about my queries.