search

AILab-CVC / SEED-Bench

(CVPR2024)A benchmark for evaluating Multimodal LLMs using multiple-choice questions.
Other
276 stars 8 forks source link

Wrong Question Types in SEED-Bench1 #28

Open littlepenguin89106 opened 1 month ago

littlepenguin89106 commented 1 month ago

I found the question type ids of "Instance Attributes" and "Instance Location" are wrong in SEED-Bench.json. Instance Location should be 3 and Instance Attribute should be 4.

Bohao-Lee commented 1 month ago

Thank you for the reminder. I have modified the json file.

Baiqi-Li commented 1 month ago

Hi, I don't know the correspondence between each question type id and the specific type such as "Instance Attributes" and "Instance Location", in which file can I see it?

Bohao-Lee commented 1 month ago

These details are displayed at the bottom of the "SEED-Bench.json" file, as the ''question_type" key in the json file.