About Whoops! dataset metric script.

chancharikmitra / CCoT

[CVPR 2024] Official Code for the Paper "Compositional Chain-of-Thought Prompting for Large Multimodal Models"

MIT License

78 stars 4 forks source link

About Whoops! dataset metric script. #9

Closed ligeng0197 closed 2 weeks ago

ligeng0197 commented 4 weeks ago

Hello, thank you very much for your creative work on scene graph based prompting. We are trying to build upon your exciting work for further research. However, we noticed that the Whoops! dataset does not provide the corresponding metric calculation script. Could you briefly introduce the method for calculating specific metrics?

Brandon3964 commented 3 weeks ago

Hi, thank you for building upon our work!

I believe Whoops provides the evaluation script under "Task Evaluation". What we used is "VQA: BEM Calculation". We slightly modified the script to make it easier to use and is attached below. Score_whoops.zip

Hope this is helpful!