Vchitect / VBench

[CVPR2024 Highlight] VBench - We Evaluate Video Generation
https://vchitect.github.io/VBench-project/
Apache License 2.0
374 stars 14 forks source link

Cannot find and process videos under provided directory #11

Closed chenerg closed 3 months ago

chenerg commented 5 months ago

I run the following vbench evaluate --videos_path ./demo/videos --dimension temporal_flickering

and get the following

args: Namespace(func=<function evaluate at 0x7f3b34fd6950>, output_path='./evaluation_results/', full_json_dir='./VBench_full_info.json', videos_path='./demo/videos', dimension='temporal_flickering', load_ckpt_from_local=None, read_frame=None)
start evaluation
Evaluation meta data saved to ./evaluation_results/temporal_flickering_full_info.json
0it [00:00, ?it/s]

It seems it cannot find the videos under the directory

yinanhe commented 5 months ago

Hi, thanks for your interest in our work! If the generated JSON does not contain any results, VBench may not be able to find the corresponding video possibly due to an incorrect sampling video name. The solution can be referenced in this issue.

When testing the temporal_flickering dimension, there should be videos with corresponding formatted names in your ./demo/videos folder. For example, with the first prompt In a still frame, a stop sign, there should be [In a still frame, a stop sign-0.mp4,..., In a still frame, a stop sign-4.mp4].

chenerg commented 3 months ago

Forget this issue, it's the same with my latest issue, same question and the answer is write new Vbench_full.json for your own data.