search

mbzuai-oryx / Video-ChatGPT

[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.
https://mbzuai-oryx.github.io/Video-ChatGPT
Creative Commons Attribution 4.0 International
1.05k stars 92 forks source link

Why TGIF videos missed and can't be processed #96

Closed FlorientHuang closed 2 months ago

FlorientHuang commented 2 months ago

Hello,

I've been exploring the TGIF Videos dataset as mentioned in this link: TGIF_DATASET However, I've encountered a couple of issues that I hope can be addressed:

Missing Videos: Upon examining the Test_frameqa_question.csv file and other annotation files, I noticed that several videos referenced are not included in the provided ZIP file.

Processing Errors: Approximately half of the videos in the dataset cannot be processed. For example, attempts to process the file at /mnt/nlp-ali/dataset/vllm/Video_ChatGPT_Dataset/Zero_Shot_QA_Dataset/TGIF-QA/videos_mp4/tumblr_nc6dweKA4W1tmwagjo1_250.mp4 resulted in errors stating "Invalid data found when processing input" and further detail on the inability to open and read the video file. Failed info: [09:54:30] /github/workspace/src/video/video_reader.cc:83: ERROR opening: /mnt/nlp-ali/dataset/vllm/Video_ChatGPT_Dataset/Zero_Shot_QA_Dataset/TGIF-QA/videos_mp4/tumblr_nhqa0w4T6e1u7c3syo1_250.mp4, Invalid data found when processing input Can't process /mnt/nlp-ali/dataset/vllm/Video_ChatGPT_Dataset/Zero_Shot_QA_Dataset/TGIF-QA/videos_mp4/tumblr_nhqa0w4T6e1u7c3syo1_250.mp4: Error reading /mnt/nlp-ali/dataset/vllm/Video_ChatGPT_Dataset/Zero_Shot_QA_Dataset/TGIF-QA/videos_mp4/tumblr_nhqa0w4T6e1u7c3syo1_250.mp4...

Could you please look into these issues? It would greatly aid in utilizing the dataset for research and development purposes.

Thank you for your assistance and for providing such a valuable resource to the community.

mmaaz60 commented 2 months ago

Hi @FlorientHuang,

Thank you for your interest in our work. It looks like the link I shared before does contain many corrupted videos. Apologies for the inconvenience.

I have downloaded the GIFs again from the original source and uploaded at google drive for your reference.

Note that the the link contain original GIFs and you may have to convert them into mp4 format before running the inference.

I hope it will help. Thanks.

FlorientHuang commented 2 months ago

Hi @FlorientHuang,

Thank you for your interest in our work. It looks like the link I shared before does contain many corrupted videos. Apologies for the inconvenience.

I have downloaded the GIFs again from the original source and uploaded at google drive for your reference.

Note that the the link contain original GIFs and you may have to convert them into mp4 format before running the inference.

I hope it will help. Thanks.

Thanks a lot

mmaaz60 commented 2 months ago

Thanks @FlorientHuang,

Good Luck and feel free to ask any further questions that you may have. Thanks