search

mbzuai-oryx / Video-ChatGPT

[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.
https://mbzuai-oryx.github.io/Video-ChatGPT
Creative Commons Attribution 4.0 International
1.05k stars 92 forks source link

Where to download the video in MSRVTT-QA & How to evaluate TGIF-QA #65

Closed wcy1122 closed 5 months ago

wcy1122 commented 8 months ago

Hello, thanks for providing this outstanding work. May I know where to download video in MSRVTT-QA? It looks like the official website [https://ms-multimedia-challenge.com/2016/dataset] is no longer maintained. Thanks

hanoonaR commented 8 months ago

Hi @wcy1122,

Can you try this website - It provides a link to the dataset. Hope this helps.

wcy1122 commented 8 months ago

Thanks for your reply. By the way, did you use the question in test_qa for MSVD-QA and MSRVTT-QA evaluation? However, did you use all questions in Test_xxx_question.csv for TGIF-QA evaluation? Thanks

wcy1122 commented 8 months ago

One more question, how to get ground-truth answer for questions in TGIF-QA? I found 5 answers in Test_xxx_question.csv files for each question, and not sure how to generate the answer for GPT 3.5 evaluation. Thanks.

KerolosAtef commented 7 months ago

I have the same questions regarding TGIF-QA , I hope anyone can reply. @hanoonaR

hanoonaR commented 6 months ago

Hi @wcy1122 ,

Thank you for your interest in our work and apologies for the delayed response. We use the validation set for MSVD and MSRVTT and the test set for TGIF. A copy of the annotation files have been attached in the links.

Please let us know if you have any further questions.

Thank you.

SCZwangxiao commented 5 months ago

Hi @wcy1122 ,

Thank you for your interest in our work and apologies for the delayed response. We use the validation set for MSVD and MSRVTT and the test set for TGIF. A copy of the annotation files have been attached in the links.

Please let us know if you have any further questions.

Thank you.

Thank you for the response. And I think better highlight that you r using TFIG-QA Frame