search

jianzongwu / MotionBooth

The official implement of research paper "MotionBooth: Motion-Aware Customized Text-to-Video Generation"
68 stars 4 forks source link

can this be extended to image to video? #4

Open SkylerZheng opened 5 days ago

SkylerZheng commented 5 days ago

wondering if this can be extended to image to video generation with image condition, and controllable camera motion?

jianzongwu commented 5 days ago

Our camera motion module is highly scalable. For vanilla text-to-video generation, you can directly use it and control the camera motion.

Since Stable Diffusion Pipeline does not provide image-to-video itself, you may need to find some image-to-video methods, like Stable Video Diffusion, and apply our camera motion method.

Please let me know if you success!