search

OpenGVLab / MMIU

MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models
https://mmiu-bench.github.io/
35 stars 1 forks source link

Please provide some example command to run the evaluation #2

Open cooleel opened 3 weeks ago

cooleel commented 3 weeks ago

Thansk for the great work. It's very intersting. Could you please provide some examples on running the evaluation? The code in VLMEvalkit folder seems incomplete.

And are you considering adding the evaluation dataset to VLMEvalkit?

FanqingM commented 2 weeks ago

Thank you, I think you should prepare the environment with the instruction of VLMEvalKit repo? We already released the whold codes, if you have prepared the environment as VLMEvalKit, I think it would not have problems.

The VLMevalKit folder in this repo is merely same with the official VLMEvalKit, with some different codes with CogVLM model. and the environment is same with the official VLMEvalKit.

For last question, yes, we will do this ASAP