search

Yushi-Hu / PromptCap

natual language guided image captioning
73 stars 7 forks source link

VQA samples for in-context learning during inference #8

Closed Artanic30 closed 10 months ago

Artanic30 commented 11 months ago

Hi, thanks for your nice work. I'm wondering what is the source of VQA samples used for in-context learning during OKVQA evaluation. Is it the VQAV2 training set, OKVQA training set or OKVQA validation set? I would appreciate it if you could provide information about my queries.

Artanic30 commented 11 months ago

image From the illustration in paper Figure4, I find these questions are from OKVQA validation set.

Yushi-Hu commented 10 months ago

Thank you for your interest in our work, and sorry for the late reply! For in-context learning, we use OK-VQA train set as in-context examples for the evaluation on OK-VQA dev set. That is the common practice for recent few shot learning works and it gives better score than examples from other dataset.

Artanic30 commented 10 months ago

Thanks for your reply. I thinks this setting is fair enough for me.