search

Vchitect / VBench

[CVPR2024 Highlight] VBench - We Evaluate Video Generation
https://vchitect.github.io/VBench-project/
Apache License 2.0
374 stars 14 forks source link

generating videos from images #13

Closed cyy-1234 closed 1 month ago

cyy-1234 commented 4 months ago

Hello, author. After reading your paper, I have some doubts about certain details. May I directly input a video clip (without any prompts) for evaluation? Is your method applicable to generating videos from images? Looking forward to your response.

yinanhe commented 4 months ago

Thank you for your interest in our work. VBench provides video quality dimensions that are relevant to the input text. You can only test relevant dimensions. [   "subject consistency",   "background consistency",   "temporal flickering",   "motion smoothness",   "aesthetic quality",   "imaging quality",   "dynamic degree",]

cyy-1234 commented 4 months ago

hello，Are there any requirements for the naming of the generated video, such as '{prompt}-{index}.mp4'? Is this format necessary?

cyy-1234 commented 4 months ago

image hi,Why do we need to subtract one?

yinanhe commented 4 months ago

hello，Are there any requirements for the naming of the generated video, such as '{prompt}-{index}.mp4'? Is this format necessary?

We have not updated the code to read frames directly for the time being, so such naming is necessary, or you can modify the code to read in frames as needed

yinanhe commented 4 months ago

image hi,Why do we need to subtract one?

When calculating the subject consistency dimension, the consistency between the current frame and the previous frame and the consistency between the current frame and the first frame are calculated, the total number of computations is num_frames -1

cyy-1234 commented 4 months ago

Hi，author， image I have successfully executed the evaluation with only video, and the results for 'background_consistency,' 'temporal_flickering,' 'motion_smoothness,' 'aesthetic_quality,' 'imaging_quality,' and 'dynamic_degree' seem to be fine. image The only area of confusion lies in the 'subject_consistency' aspect，Two numerical values are inconsistent. image

cyy-1234 commented 4 months ago

image image image

Hi, when I run multiple videos at once to calculate aesthetic_quality, I always get the error 'CUDA out of memory.' What could be the reason? According to my understanding, the model should only load one video at a time, and the size of the input videos is small.

yinanhe commented 4 months ago

This does not seem normal, you need to check your local environment

ziqihuangg commented 2 months ago

Hello, author. After reading your paper, I have some doubts about certain details. May I directly input a video clip (without any prompts) for evaluation? Is your method applicable to generating videos from images? Looking forward to your response.

@cyy-1234 Hi, please check out VBench-I2V for evaluating image-to-video models: https://github.com/Vchitect/VBench/tree/master/vbench2_beta_i2v

ziqihuangg commented 1 month ago

Hi, we're closing this issue as it appears your questions have been addressed. However, feel free to open a new issue or reopen this one, if you have further questions or if anything else comes up related to this issue.

Crazy-wu-20 commented 3 weeks ago

image image image

Hi, when I run multiple videos at once to calculate aesthetic_quality, I always get the error 'CUDA out of memory.' What could be the reason? According to my understanding, the model should only load one video at a time, and the size of the input videos is small.

Hi, have you solved this question? I have the same problem.