search

EvolvingLMMs-Lab / lmms-eval

Accelerating the development of large multimodal models (LMMs) with one-click evaluation module - lmms-eval.
https://lmms-lab.framer.ai/
Other
2.09k stars 157 forks source link

mvbench-missing videos #403

Open orrzohar opened 2 weeks ago

orrzohar commented 2 weeks ago

AssertionError: video path not found

~/.cache/huggingface/mvbench_video/sta/sta_video_segment/NBMH9_21.4_37.6.mp4

seems like some videos are missing in mvbench, making evaluation impossible.

kcz358 commented 1 week ago

Hi @yinanhe , can you help look into this?

yinanhe commented 1 week ago

@orrzohar you can find the missing video here~ https://huggingface.co/datasets/OpenGVLab/MVBench/blob/main/video/data0613.zip