search

ltdrdata / ComfyUI-Impact-Pack

Custom nodes pack for ComfyUI This custom node helps to conveniently enhance images through Detector, Detailer, Upscaler, Pipe, and more.
GNU General Public License v3.0
1.91k stars 187 forks source link

AnimateDiff/Batch won't work with Controlnet SEGS and Mask to SEGS (Combined) #736

Open oliverban opened 2 months ago

oliverban commented 2 months ago

Hello!

First off, thanks so much for a wonderful extension pack!

I am trying to get this head mask I made in 3D to become a SEGS for a "Detailer for AnimateDiff" but it keeps failing to see the segs. First off it takes an enormous amount of time to even go through the "Controlnet Apply (SEGS)" and once it does the image preview on the Detailer is all wrong, showing a different all skewed part of the frame.

Attached in a screenshot. The only thing coming in on the LEFT is the original video that I want to fix the face on. The video coming in at the top is just a green mask of the head going to a Mask from Color and then into a Combined MASK TO SEGS.

I KNOW this should work and that it has something to do with LISTS and BATCHES but I am at my wits end! Thanks for all the help!

FYI: I tested this exact setup (but normal Detailer, not Animatediff Version) and it worked perfectly! :) I just want it to work with AnimateDiff/Batches as well!

brave_JmRwWVR71h
oliverban commented 2 months ago

Bump? :) Or is it not doable?

ltdrdata commented 2 months ago

Have you been read this description?

https://github.com/ltdrdata/ComfyUI-extension-tutorials/blob/Main/ComfyUI-Impact-Pack/tutorial/detectors.md#simple-detector-for-animatediff

oliverban commented 2 months ago

Have you been read this description?

https://github.com/ltdrdata/ComfyUI-extension-tutorials/blob/Main/ComfyUI-Impact-Pack/tutorial/detectors.md#simple-detector-for-animatediff

Yes I have! But it doesn't say anything on using your own masks, only describes how to use the detector one with BBOX? I want to use my own masks instead since the Simple Detector also fails on detecting a lot of the frames. I was hoping to be able to use the Mask to SEGS one.

ltdrdata commented 2 months ago

image Do you know this node?

The AnimateDiff version differs in whether it handles the mask as a batch or not. The mask contained in a single SEGS becomes a mask with the same batch size as the input batch image.

oliverban commented 2 months ago

image Do you know this node?

The AnimateDiff version differs in whether it handles the mask as a batch or not. The mask contained in a single SEGS becomes a mask with the same batch size as the input batch image.

Hmm, haven't seen the one with AnimateDiff attached. I will try it out. Thanks! <3