[Feature]: New Update! 👀

[Mistermango23]

Expected behavior

Look at this! a new Version of animatediff V3 https://github.com/guoyww/AnimateDiff @continue-revolution !!!!!!!

[continue-revolution]

Wait, wait, wait. I’ve been working full night for my compiler assignment for 2 days. If you want to see more update instead of my obituary, you will have to be more patient.

[dancemanUK]

Wait, wait, wait. I’ve been working full night for my compiler assignment for 2 days. If you want to see more update instead of my obituary, you will have to be more patient.

谢谢你的努力，我们不急。

[continue-revolution]

After partial investigation of the update -

1. Supporting new motion module will very easy. Change to the repo would be minimal
2. Supporting new adapter (lora) will also be very easy, but I need to investigate the difference between motion lora and domain adapter
3. Having a good user experience of ControlNet while having a not-too-bad implementation (meaning that the code duplicate would be minimal) is always very challenging, so do the new control models from AnimateDiff researchers

So here is the plan -

• In v1.13.0 which may come today or tomorrow, you will be able to use new motion module and new adapter. Code change will only happen in this repository.
• In v2.0.0 which may come sometime next week, you will be able to use new control models as well as magic animate through ControlNet panel. Code change will mainly happen in sd-webui-controlnet (I also have write permission there).

The estimated release time is tentative. Factors such as my health condition may delay the development again.

If you are curious about SVD from SAI: supporting this model is 100 times more challenging than having ControlNet or anything else because A1111 WebUI is a huge repo and I need to consider much more factors than having an extension like this. I cannot give an estimate when you can have SVD in A1111, but I am pretty sure that the code change will happen in A1111 WebUI. AnimateDiff will AT MOST benefit from the VAE in SVD.

[armynew]

waiting for new version release,thank you !

[continue-revolution]

v1.13.0 has been released, which means that V3 motion module and V3 domain adapter have been supported. Make sure to read https://github.com/continue-revolution/sd-webui-animatediff?tab=readme-ov-file#v3.

This issue will be closed when v2.0.0 is released, which means that SparseCtrl and Magic Animate are supported.

[LIQUIDMIND111]

v1.13.0 has been released, which means that V3 motion module and V3 domain adapter have been supported. Make sure to read https://github.com/continue-revolution/sd-webui-animatediff?tab=readme-ov-file#v3.

This issue will be closed when v2.0.0 is released, which means that SparseCtrl and Magic Animate are supported.

ok , so after reading, im confuse in this: v3 model goes to the MODEL animatediff folder, and v3 ADAPTER goes in the LORA folder? and needs to be ACTIVATED as a LORA?

[continue-revolution]

@LIQUIDMIND111 yes. Adapter is optional.

[LIQUIDMIND111]

@LIQUIDMIND111 yes. Adapter is optional.

ok im testing it already and just seems that MOTION loras from v2 give error when you activate them using base v3. Is this normal since motion loras are for v2 only?

NOTE: Seems quality its better when NOT using v3 adapter as a LORA..... i do have it on the LORA folder, but not activating it like a motion lora...

[continue-revolution]

V3 does not have mid block injection but V2 does. Motion Lora also applied to mid block motion modules, so they are not compatible.

[LIQUIDMIND111]

V3 does not have mid block injection but V2 does. Motion Lora also applied to mid block motion modules, so they are not compatible.

got you... 👯

[LIQUIDMIND111]

V3 does not have mid block injection but V2 does. Motion Lora also applied to mid block motion modules, so they are not compatible.

hey friend, one question, FOR COMFY UI, do we need to use other models for animate diff? or can we use the same we use for auto1111 like mm1.5_v2_safetensor?

[continue-revolution]

You can use motion module there, but I’m not sure how they implement adapter.

[LIQUIDMIND111]

You can use motion module there, but I’m not sure how they implement adapter.

ok, i already tried, works ok, maybe auto111 works better.... v3 works well with LCM in auto1111 ... look.

https://github.com/continue-revolution/sd-webui-animatediff/assets/112510084/cd329550-3d88-439d-868e-1e7d21e6583e

[continue-revolution]

SparseCtrl is already available in dev branch and it's working properly. I will close this, though v2.0.0 will be released a few days later after I've done with magic animate and a bunch of side features.

[m20renderz]

You can use motion module there, but I’m not sure how they implement adapter.

ok, i already tried, works ok, maybe auto111 works better.... v3 works well with LCM in auto1111 ... look.

00009-3336737030.mp4

LCM as in sampler or LORA? which v3 module is this , please share workflow, thanks