search

AILab-CVC / SEED-Bench

(CVPR2024)A benchmark for evaluating Multimodal LLMs using multiple-choice questions.
Other
315 stars 12 forks source link

Wrong Question Types in SEED-Bench1 #28

Open littlepenguin89106 opened 6 months ago

littlepenguin89106 commented 6 months ago

I found the question type ids of "Instance Attributes" and "Instance Location" are wrong in SEED-Bench.json. Instance Location should be 3 and Instance Attribute should be 4.

Bohao-Lee commented 6 months ago

Thank you for the reminder. I have modified the json file.

Baiqi-Li commented 5 months ago

Hi, I don't know the correspondence between each question type id and the specific type such as "Instance Attributes" and "Instance Location", in which file can I see it?

Bohao-Lee commented 5 months ago

These details are displayed at the bottom of the "SEED-Bench.json" file, as the ''question_type" key in the json file.