search

ltdrdata / ComfyUI-extension-tutorials

579 stars 32 forks source link

Face detailer for animatediff resulting in black mask. #30

Open amnesicloud opened 9 months ago

amnesicloud commented 9 months ago

Hi there, thanks for the tutorials, it has been a great help. I'm having problem when making modifications based on your workflow: ComfyUI-Impact-Pack/workflow/detailer-for-animatediff.png https://github.com/ltdrdata/ComfyUI-extension-tutorials/blob/Main/ComfyUI-Impact-Pack/workflow/detailer-for-animatediff.png

I'm using this gif as input: input

and this is my workflow, where I removed the first Ksampler node since I'm using specific input: test_workflow (1).json

and my result are as follow: AnimateDiff_00052

where the face is black masked. I'm wondering what's causing this problem?

amnesicloud commented 9 months ago

to me it has successfully detected the face, but failed to redraw, but I can't make it work

ltdrdata commented 9 months ago

What is your gpu?

amnesicloud commented 9 months ago

using Apple Silicon M2, I use comfyui with this command: ./venv/bin/python main.py --force-fp16

amnesicloud commented 9 months ago

with 32G Mem

ltdrdata commented 9 months ago

Show me the full log comfyui.log

My guess is that M2 is related to issues occurring when performing ksampling at a specific resolution.

amnesicloud commented 9 months ago

update: I adopted the method from https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved/issues/48#issuecomment-1750156332, and the black mask problem is gone. However now the face part is weird images, it seems AnimateDiff_00068

the log is as follow comfyui.log

ltdrdata commented 9 months ago

Before proceeding further, it is recommended that you update your ComfyUI. It's already a month old version.

amnesicloud commented 9 months ago

I have updated ComfyUI, and I get rid of previous update, it causes problem in animatediff workflow comfyui.log

current result is like this: AnimateDiff_00092

ltdrdata commented 9 months ago

Try creating images in 640, 674 resolution in the basic AnimateDiff workflow without FaceDetailer, if that's not an issue, apply i2i.

amnesicloud commented 9 months ago

I'm not sure what you mean i2i, image2image? I have "basic AnimateDiff workflow without FaceDetailer" working, and I've tried to apply face detailer on each frame (is that what you meant by "i2i"?), however that would cause the face flickering, that's why I'm looking for an animatediff detailer solution.

ltdrdata commented 9 months ago

I meant vid2vid. Just make source image size as 640, 674.