search

Kosinkadink / ComfyUI-AnimateDiff-Evolved

Improved AnimateDiff for ComfyUI and Advanced Sampling Support
Apache License 2.0
2.37k stars 180 forks source link

MotionLoRA Training #290

Open ExponentialML opened 5 months ago

ExponentialML commented 5 months ago

Hello, thanks for your work! This isn't really an issue, but more of an enhancement.

I've just released training code for MotinLoRAs that are compatible with this repository. It uses the recently released MotionDirector method as it decouples the spatial information from the temporal, allowing you to focus more on motion modelling. You can try out the code here:

https://github.com/ExponentialML/AnimateDiff-MotionDirector

Here's an example from a recent run (at default settings from my repository) that took about 13.5GB of VRAM and trained in about 300 steps in 10 minutes on a 3090.

sample-300

Edit: Added update for 500 steps in 16 minutes of training

sample-500

I saw in the README that training MotionLoRAs are in the works, so this may introduce some useful synergy between these two repositories :+1: .

Kosinkadink commented 5 months ago

Thanks for the info and the code! I'll definitely check it out, great work!