search

AILab-CVC / SEED-Bench

(CVPR2024)A benchmark for evaluating Multimodal LLMs using multiple-choice questions.
Other
276 stars 8 forks source link

What is the correct way to download the video #17

Closed teasgen closed 6 months ago

teasgen commented 6 months ago

Hi! Thank you for your benchmark! I'm trying to download v1_video.zip.001 from https://huggingface.co/datasets/AILab-CVC/SEED-Bench. How should I correctly get the frames? I can't just unzip the file, because of error this file is not a zipfile.

Bohao-Lee commented 6 months ago

Thank you for your interest in our project. Please download all the files named v1_video.zip.***. Afterward, cat them and unzip with the extraction.

teasgen commented 6 months ago

Thank you for your interest in our project. Please download all the files named v1_video.zip.***. Afterward, cat them and unzip with the extraction.

I download videos as you mentioned, but some of frames are incorrect (I found only one). For example, 4.png from task10/ssv2_8_frame/v833. Could you please check it?

Bohao-Lee commented 6 months ago

Thank you very much for your correction. We have updated the corresponding file. Please check SEED-Bench-1 for detailed.

teasgen commented 6 months ago

Thank you very much for your correction. We have updated the corresponding file. Please check SEED-Bench-1 for detailed.

Unfortunately, there are still corrupted frames. Full list: v4060, v4078, v2290, v2251 (all 4.png). v786, v909, v1283, v3218 (all 5.png). I suppose there are other broken files, could you please check them all? On my pc all of these images look like empty files without any annotations or other information.

Bohao-Lee commented 6 months ago

I apologize for any inconvenience caused. We have re-uploaded a version of the data and utilized the PIL library to verify the validity of each image prior to uploading. Please check SEED-Bench-1.

teasgen commented 6 months ago

I apologize for any inconvenience caused. We have re-uploaded a version of the data and utilized the PIL library to verify the validity of each image prior to uploading. Please check SEED-Bench-1.

Everything is OK now! Many thanks for quick support