mme-real-world options - Githubissues

open-compass / VLMEvalKit

Open-source evaluation toolkit of large vision-language models (LVLMs), support 160+ VLMs, 50+ benchmarks

https://huggingface.co/spaces/opencompass/open_vlm_leaderboard

Apache License 2.0

1.34k stars 188 forks source link

mme-real-world options #452

Closed yfzhang114 closed 2 months ago

yfzhang114 commented 2 months ago

Updated the construction of options for creating CSV files in MMERealWorld, and fixed issues where some models were unable to correctly load prompts.

yfzhang114 commented 2 months ago

Updated the scoring method in MME-RealWorld to align with the original version.

Mor-Li commented 2 months ago

Hi @yfzhang114. It seems the lint check didn’t pass. Could you kindly run the pre-commit hooks when you get a chance? Thanks so much for your time and effort!

yfzhang114 commented 2 months ago

Hi @yfzhang114. It seems the lint check didn’t pass. Could you kindly run the pre-commit hooks when you get a chance? Thanks so much for your time and effort!

fixed.