search

open-compass / VLMEvalKit

Open-source evaluation toolkit of large vision-language models (LVLMs), support 160+ VLMs, 50+ benchmarks
https://huggingface.co/spaces/opencompass/open_vlm_leaderboard
Apache License 2.0
1.4k stars 195 forks source link

[Feat] add modelscope download video datasets #623

Closed Yunnglin closed 1 day ago

Yunnglin commented 4 days ago

Set env VLMEVALKIT_USE_MODELSCOPE=True and pip install modelscope

you can download video benchmarks from modelscope automatically, which supports:

all dataset downloads have been tested.

kennymckormick commented 1 day ago

To be compatible with more users conventions, you can now set VLMEVALKIT_USE_MODELSCOPE to 1 or True