search

Vchitect / VBench

[CVPR2024 Highlight] VBench - We Evaluate Video Generation
https://vchitect.github.io/VBench-project/
Apache License 2.0
506 stars 26 forks source link

Evaluation of True Videos #17

Closed chenerg closed 6 months ago

ziqihuangg commented 6 months ago

Hi, thanks for your interest in our work. Could you let us know which step you get stuck at? We are happy to assist.

chenerg commented 6 months ago

Yes, after download the VBench_full_info.json (not edited and I don't know what the content means), then I run vbench evaluate --videos_path ./videos/ --dimension "imaging_quality", Then I got site-packages/vbench/imaging_quality.py", line 20, in technical_quality average_score = sum([o['video_results'] for o in video_results]) / len(video_results) ZeroDivisionError: division by zero

The reason is because the video_list inputed in the following site-packages/vbench/imaging_quality.py", line 33, in compute_imaging_quality all_results, video_results = technical_quality(model, video_list, device) is empty.

But under the given path ./videos/, there are hundreds of videos, I can't solve this.

ziqihuangg commented 6 months ago

Can you provide ls ./videos?

chenerg commented 6 months ago

OK, I know, it's about writing your own VBench_full.json ; My goal is not to evaluate AIGC generated videos but true videos, so I rewrite VBench module and it worked.