Does every new animation need to be retrained?

showlab / MotionDirector

[ECCV 2024 Oral] MotionDirector: Motion Customization of Text-to-Video Diffusion Models.

https://showlab.github.io/MotionDirector/

Apache License 2.0

853 stars 54 forks source link

Does every new animation need to be retrained? #35

Open yize0106 opened 7 months ago

yize0106 commented 7 months ago

Hello, Thanks for releasing the nice work. I have just come into contact with this field, and I have a question that I am curious about: If I want to animate new image with different appearances and customize new actions, do I need to use the image and related videos to train again?

If so, does it mean that the weights given by link were trained only on the reference image/video and no other new images/videos were used?

Looking forward to your answer very much！

ruizhaocv commented 3 months ago

Yes, for new motions, you need to train the LoRAs again. This is a typical setting in the tuning-based customization task.