search

open-compass / VLMEvalKit

Open-source evaluation toolkit of large vision-language models (LVLMs), support ~100 VLMs, 40+ benchmarks
https://huggingface.co/spaces/opencompass/open_vlm_leaderboard
Apache License 2.0
1.08k stars 154 forks source link

About Tablevqabench score #442

Open mary-0830 opened 3 weeks ago

mary-0830 commented 3 weeks ago

Hello, may I ask which scores the TableVQAbench output should correspond to in the paper? image

image And where should Input Modalities be set?

Thanks for your reply~

FangXinyu-0913 commented 1 week ago

Hi @mary-0830, For TableVQABench, as of now VLMEvalKit only supports evaluating MLLMs, and the output modal only supports Vision, the specific four values that correspond to the four values of the same name on the table. The calculation of the average value is a bit off from my previous knowledge, maybe ask @hkunzhe.