search

AILab-CVC / VideoCrafter

VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models
https://ailab-cvc.github.io/videocrafter2/
Other
4.58k stars 342 forks source link

Dose detailed caption improve the performance? #81

Open g-jing opened 4 months ago

g-jing commented 4 months ago

I have a few questions about the video caption.

  1. I noticed that during the training, the caption in the video CSV is quiet short. Will the performance improve if we use a detailed caption during the test time?
  2. If we also use detailed caption during the training time, will that improve the model performance?
  3. The caption used now focuses more on the frame description rather than the video dynamics. Will that be improved with a capture describing the dynamics? If so, do you have any suggestions on generating that captions?

Thanks a lot!