evalcrafter / EvalCrafter

[CVPR 2024] EvalCrafter: Benchmarking and Evaluating Large Video Generation Models
http://evalcrafter.github.io
115 stars 7 forks source link

Could you provide VideoCrafter2 Raw metrics? #12

Closed daeunni closed 1 month ago

daeunni commented 3 months ago

Hi, thanks for your nice work and communication! Could you provide VideoCrafter2 raw metrics like below? (Not average per each metric category) Thanks :)

image
daeunni commented 3 months ago

I already checked this link which can download generated videos from VideoCrafter2.

Could you share the exact random seed? or specific parameters below?

CUDA_VISIBLE_DEVICES=3 python3 scripts/evaluation/inference.py \
--seed 123 \
--mode 'base' \
--ckpt_path $ckpt \
--config $config \
--savedir $res_dir/$name \
--n_samples 1 \
--bs 3 --height 320 --width 512 \
--unconditional_guidance_scale 12.0 \
--ddim_steps 50 \
--ddim_eta 1.0 \
--prompt_file $prompt_file \
--fps 28 \
--reverse

Since from my side, I cannot reimplement VideoCrafter2's performance with seed 123. I used this checkpoint from huggingface the author provided.

Yaofang-Liu commented 3 months ago

Hi, we used VideoCrafter2 of 640*1024 resolution, and I will upload the corresponding raw metrics later.

Yaofang-Liu commented 3 months ago

Please find the raw matric values for VideoCrafter2 here.