[Bug]: AnimateDiff & ADetailer incompatibility - [32, 2560, 12, 8]

[tutu329]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Have you read FAQ on README?

• [X] I have updated WebUI and this extension to the latest version

What happened?

只要打开adetailer，ainimatediff生成一次视频后，必报此错，然后必须重启sd webui才行。 不打开adetailer生成视频后没有任何问题，但是因为脸部没有有效修复，视频还是不可用。

Steps to reproduce the problem

1. 打开adetailer
2. 打开animatediff
3. 生成视频

What should have happened?

RuntimeError: Expected weight to be a vector of size equal to the number of channels in input, but got weight of shape [1280] and input of shape [32, 2560, 12, 8]

Commit where the problem happens

webui: 1.6 extension: animatediff

What browsers do you use to access the UI ?

No response

Command Line Arguments

--opt-sdp-attention --api --server 0.0.0.0 --port 5000 --enable-insecure-extension-access --ad-no-huggingface --api-server-stop

Console logs

RuntimeError: Expected weight to be a vector of size equal to the number of channels in input, but got weight of shape [1280] and input of shape [32, 2560, 12, 8]

Additional information

No response

[tutu329]

如果这个bug解决了，animatediff插件真的无敌了！

[continue-revolution]

我暂时不打算支持adetailer。你们一般用adetailer干啥，如果有可能的话我可以在我的另一个插件SegmentAnything里面支持这个。

[GuruVirus]

我用animatediff运行了adetailer，但你不想这样做。 adetailer 的种子（？）不会保持不变，因此脸部只会随着每一帧而变化。

[continue-revolution]

@GuruVirus It seems that you are not native Chinese speaker. If that's the case, feel free to speak English.

[tutu329]

我暂时不打算支持adetailer。你们一般用adetailer干啥，如果有可能的话我可以在我的另一个插件SegmentAnything里面支持这个。

倒也不是一定要用adetailer，主要在非特写镜头中，脸部通常完全没法看，adetailer就是解决非特写镜头的脸部恢复问题（对脸部进行重绘。sd自带的脸部恢复效果还是非常不好。），用了几个插件似乎只有adetailer能比较好的恢复脸部。不知大神有没有其他支持的脸部恢复插件。

[continue-revolution]

理论上脸部恢复仅仅只是目标检测面部然后局部重绘，我觉得我写的SegmentAnything插件可以达到同样的效果。但是目前那个插件只能手动重绘，我听说adetailer的整个过程都是自动化了。

[tutu329]

我用animatediff运行了adetailer，但你不想这样做。 adetailer 的种子（？）不会保持不变，因此脸部只会随着每一帧而变化。

adetailer重绘脸部确实会造成脸部不稳定，不过我主要将adetailer用于中远镜的人物视频（因为非特写镜头中人物脸部通常丑的完全无法看），既然是中远镜头，脸部完全无法看和脸部有一些不稳定，我绝对选择后者。而且animatediff插件是支持adetailer的，能正常出视频，只是生成视频后哪个webui的环境变量可能没有重置好，导致了只能生成一次。如果revolution大神能修复这个小bug将不胜感激！

[tutu329]

理论上脸部恢复仅仅只是目标检测面部然后局部重绘，我觉得我写的SegmentAnything插件可以达到同样的效果。但是目前那个插件只能手动重绘，我听说adetailer的整个过程都是自动化了。

adetailer是自动检测脸部区域和脸的数量，然后自动重绘的。目前大神你的animatediff插件是支持adetailer的，能正常出视频，只是生成视频后可能哪个webui的环境变量没有重置好，导致了只能生成一次。如果revolution大神能修复这个小bug将不胜感激！

[njmp11]

adetailer is great for still images but not really for sequences. You can test more rapidly by using img2img mode on animatediff sequences (to avoid restarting between every run, as you would in t2i animatediff + adetailer) and you will see it does not live up to the temporal coherence of the original animatediff sequence. Especially when using prompt travel in animatediff, adetailer breaks many things like character facial expressions changing. If there were some way to do a similar inpaint in animatediff but maintaining the temporal coherence and stability, that would be excellent.

I think the adetailer crash must be fixed because it is a serious problem to have to restart webui every time adetailer is used before animatediff, but I think we ultimately will need a face fixer specialized in stable sequences. adetailer is not the solution here (much as I love it for stills).

[continue-revolution]

I will try to develop a solution within my Segment Anything extension since that extension is under my full control. However, since I have multiple responsibilities in real life, you will have to wait patiently.

ADetailer is not under my control - I've neither read its source code nor know what it did that break things up. If you open adetailer, then close adetailer, then use animatediff, then your webui crush, I believe that's the problem of adetailer - it failed to restore to the original state after it finish its role. If that is the case, you may want to file a issue there.

[continue-revolution]

If you open adetailer, then forgot to close adetailer, then use animatediff, then the crush is expected, not the problem of adetailer.

[StellineOrb]

我暂时不打算支持adetailer。你们一般用adetailer干啥，如果有可能的话我可以在我的另一个插件SegmentAnything里面支持这个。

继续革命大大，Adetailer和你的segement everything我都用，但其实用的场合不同，Adetailer用来批量精细化脸部效率很高，就是点一下，prompt都和上面一样，顺带便但提升很大的步骤，就像上面的兄弟说的，不用Adetailer很多全身像脸是没法看的。无论是现在不支持要后期手调，还是再单独开一次segment everything，都会增加一个working progress。希望考虑下增加对Adetailer的支持，实用性会好很多，不然生成内容只敢维持在半身像了！

[StellineOrb]

理论上脸部恢复仅仅只是目标检测面部然后局部重绘，我觉得我写的SegmentAnything插件可以达到同样的效果。但是目前那个插件只能手动重绘，我听说adetailer的整个过程都是自动化了。

是的，其实想要的就是这个自动化！

[continue-revolution]

@StellineOrb 我知道他底层是干啥的，他本质上就是把你的图发到i2i然后inpaint，如果我搞一个前后段交互一下可能可以做出这个feature，但是像他那样自己调StableDiffusionImg2imgProcess我估计搞不了。

[StellineOrb]

@StellineOrb 我知道他底层是干啥的，他本质上就是把你的图发到i2i然后inpaint，如果我搞一个前后段交互一下可能可以做出这个feature，但是像他那样自己调StableDiffusionImg2imgProcess我估计搞不了。

是的，就是inpaint。从用户的角度来分析下：对我来讲达到类似效果有多种办法，但是我会选流程最短，施加人工控制最少的那种，现在我animatediff用下来的感觉如果mov2mov完我还得抽帧调img2img不如从一开始我就用批量img2img的方式再通过ebsynth之类的优化，这样img2img的过程中脸部自动就优化了，流程就少一步。供参考。

[AiGirlfriend555]

i keep getting similar error too, at some point i tried to remove all the embedding/textual inversion (etc: badhandv4,easynegative) that was in my negative prompt and i no longer receive error. many ppl claim to be adetailer being the issue but it's really not. i am using adetailer but i don't get any error anymore.

[continue-revolution]

I believe that it is compatible now.