search

open-compass / VLMEvalKit

Open-source evaluation toolkit of large vision-language models (LVLMs), support 160+ VLMs, 50+ benchmarks
https://huggingface.co/spaces/opencompass/open_vlm_leaderboard
Apache License 2.0
1.34k stars 188 forks source link

MME-RealWorld Not Supported. #449

Closed Haochen-Wang409 closed 2 months ago

Haochen-Wang409 commented 2 months ago

Hello, I found that MME-RealWorld has not been supported:

ERROR - Dataset MME-RealWorld is not valid, will be skipped.
PhoenixZ810 commented 2 months ago

Hi, this problem has been fixed just now. Thank you!

Haochen-Wang409 commented 2 months ago

Thanks for the prompt reply! The fix works for me.