search

open-compass / VLMEvalKit

Open-source evaluation toolkit of large vision-language models (LVLMs), support ~100 VLMs, 40+ benchmarks
https://huggingface.co/spaces/opencompass/open_vlm_leaderboard
Apache License 2.0
1.1k stars 157 forks source link

[Request]Consider integrating MMT-Bench and CONTEXTUAL? #196

Closed iamlockelightning closed 2 months ago

iamlockelightning commented 4 months ago

MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI https://arxiv.org/pdf/2404.16006

CONTEXTUAL: Evaluating Context-Sensitive Text-Rich Visual Reasoning in Large Multimodal Models https://arxiv.org/pdf/2401.13311

KainingYing commented 4 months ago

Thanks for your interest in MMT-Bench. We will integrate MMT-Bench ASAP.

kennymckormick commented 3 months ago

Hi, @iamlockelightning MMT-Bench has already been supported.