search

rese1f / MovieChat

[CVPR 2024] MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
https://rese1f.github.io/MovieChat/
BSD 3-Clause "New" or "Revised" License
524 stars 41 forks source link

Evaluation json file #11

Closed wanghao-cst closed 1 year ago

wanghao-cst commented 1 year ago

Hi, may I know what is the file used in "--gt_file"? Is it train, test or val.json of MSVD-QA? I want to replicate the result shown in paper. python run_inference_qa_msvd.py \ --cfg-path eval_configs/MovieChat.yaml \ --gpu-id 0 \ --num-beams 1 \ --temperature 1.0 \ --video-path /path/to/your/video \ --gt_file /path/to/your/question and answer file \ --output_dir /path/to/your/output \ --output_name msvd-qa \ --fragment-video-path src/video_fragment/output.mp4 \

rejoicelf commented 1 year ago

I download it from this repo, https://github.com/antoyang/just-ask. It has MSVD-QA, MSRVTT-QA and ActivityNet-QA. Maybe some preprocess code need be to wtrite by your own. Thank you for attention.

rejoicelf commented 1 year ago

Hi, may I know what is the file used in "--gt_file"? Is it train, test or val.json of MSVD-QA? I want to replicate the result shown in paper. python run_inference_qa_msvd.py --cfg-path eval_configs/MovieChat.yaml --gpu-id 0 --num-beams 1 --temperature 1.0 --video-path /path/to/your/video --gt_file /path/to/your/question and answer file --output_dir /path/to/your/output --output_name msvd-qa --fragment-video-path src/video_fragment/output.mp4 \

It is the test_qa.json of MSVD-QA, Same as MSRVTT_QA and ActivityNet-QA. All is the test split.

wanghao-cst commented 1 year ago

Hi, may I know what is the file used in "--gt_file"? Is it train, test or val.json of MSVD-QA? I want to replicate the result shown in paper. python run_inference_qa_msvd.py --cfg-path eval_configs/MovieChat.yaml --gpu-id 0 --num-beams 1 --temperature 1.0 --video-path /path/to/your/video --gt_file /path/to/your/question and answer file --output_dir /path/to/your/output --output_name msvd-qa --fragment-video-path src/video_fragment/output.mp4 \

It is the test_qa.json of MSVD-QA, Same as MSRVTT_QA and ActivityNet-QA. All is the test split.

Thank you for your help~