search

Doubiiu / DynamiCrafter

[ECCV 2024] DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors
Apache License 2.0
2.09k stars 165 forks source link

Is it possible to generate video with fixed pose? #38

Closed guanjunwu closed 3 months ago

guanjunwu commented 3 months ago

Hi, thanks for your great work! I wonder if I can use DynamiCrafter to generate the video with the fixed pose, such as an application like 2.2 Looping video Generation I wish the camera was fixed and no changes in the camera and we could only focus on the scene's motion. How can we design the prompt?(choose image, and text prompt)

Doubiiu commented 3 months ago

Hi. Thanks for your recognition. That's an interesting question. Currently, I think it is hard to fix the camera motion, as it is a part of the dynamic learned from the conditional image context. The prompt like rotating view will make the corresponding camera motion. If you don't give any camera motion in the prompt, it may mostly produce still camera motion. However, I am not sure if a prompt like fixed camera works for your case. It is hard to say. From my point of view, fine-tuning the model on a specific dataset with still camera motion for 10k steps (maybe?) may work.