search

Doubiiu / DynamiCrafter

[ECCV 2024, Oral] DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors
https://doubiiu.github.io/projects/DynamiCrafter/
Apache License 2.0
2.56k stars 205 forks source link

About the loss #128

Open LeningWang opened 1 month ago

LeningWang commented 1 month ago

How does the loss function behave, and after how many steps train can we generate an image instead of noise? Thanks!

Doubiiu commented 1 month ago

Hi, since we build our model based on pre-trained t2v model VideoCrafter. it can already generate videos at the beginning.

LeningWang commented 1 month ago

Thank you! If I use the base architecture of DynamiCrafter to train on our own dataset without using the existing pre-trained parameters, is that feasible? Approximately how many steps would the training take? Thanks again!

Doubiiu commented 1 month ago

Emmm I am not sure. Usually, we will train models on pre-trained weights to make our life easier.