search

Vchitect / VBench

[CVPR2024 Highlight] VBench - We Evaluate Video Generation
https://vchitect.github.io/VBench-project/
Apache License 2.0
374 stars 14 forks source link

LeaderBoard code #25

Closed GFENGG closed 1 month ago

GFENGG commented 2 months ago

Hello, Do you have a plan to release the code for the LeaderBoard?

ziqihuangg commented 2 months ago

Leaderboard code is available at https://huggingface.co/spaces/Vchitect/VBench_Leaderboard/tree/main.

GFENGG commented 2 months ago

Leaderboard code is available at https://huggingface.co/spaces/Vchitect/VBench_Leaderboard/tree/main.

hello, i find the necessary files under this code are missed, Such as: CSV_DIR = "./vbench_leaderboard_submission/results.csv" QUALITY_DIR = "./vbench_leaderboard_submission/quality.csv"

Where can i find these files?

ziqihuangg commented 2 months ago

These results are already displayed in the leaderboard. Thanks.

GFENGG commented 2 months ago

These results are already displayed in the leaderboard. Thanks.

Hello, I find some dimensions are missed in the upload sampled videos, for example, videocrafter-1.tar.gz don not have the dimension of "background consistency" So the results of the online LeaderBoard can not be reproduced exactly.

ziqihuangg commented 2 months ago

Hi, for missing videos: Some dimensions, such as "background consistency" and "scene," use the same set of videos.

plus: It would be great if you open new issues for each question or email us for ongoing discussions. Thanks!

GFENGG commented 2 months ago

Hi, for missing videos: Some dimensions, such as "background consistency" and "scene," use the same set of videos.

plus: It would be great if you open new issues for each question or email us for ongoing discussions. Thanks!

Thanks，i will try with the same video sets，and i will open a new issue if i have other questions