Before you open a pull-request, please check if a similar issue already exists or has been closed before.
When you open a pull-request, please be sure to include the following
[ ] A descriptive title: [xxx] XXXX
[ ] A detailed description
Thank you for your contributions!
I have updated II-Bench in your repository, setting it to the "none" configuration as per the paper. Testing with idefics2-8b shows results that are largely consistent with those in the paper (test: 68.12%, paper: 67.69%). The dataset has not yet been uploaded to lmms-lab, so please fork the dataset from https://huggingface.co/datasets/m-a-p/II-Bench.
Before you open a pull-request, please check if a similar issue already exists or has been closed before.
When you open a pull-request, please be sure to include the following
Thank you for your contributions!
I have updated II-Bench in your repository, setting it to the "none" configuration as per the paper. Testing with idefics2-8b shows results that are largely consistent with those in the paper (test: 68.12%, paper: 67.69%). The dataset has not yet been uploaded to lmms-lab, so please fork the dataset from https://huggingface.co/datasets/m-a-p/II-Bench.