cubiq / ComfyUI_InstantID

Apache License 2.0
944 stars 50 forks source link

Faceswap workflow #98

Open skyler14 opened 3 months ago

skyler14 commented 3 months ago

There was a custom node library preceding the release this one called comfyui instantid faceswap, which seemed to do a type of inpainting based around InstantID using any model (not limited to inpainting). Is it possible to recreate this with an workflow in this UI or are there missing modules that his use of the diffusers library allowed. Have you experimented with any inpainting pipelines and gooten similar results and if so can you add those to examples or attach here?

cubiq commented 3 months ago

inpainting is totally possible, the workflow is the same as standard inpainting. I'll post a workflow when I have time

Kwisss commented 3 months ago

I get this error with in-painting models I made with comfy ((inpaint_model - base_model) * 1.0 + other_model):

RuntimeError: shape '[320, 9, 3, 3]' is invalid for input of size 11520

Edit: 11520 seems to be the tensor shape of [320,4,3,3], is it even possible to fit it with a differently shaped Tensor? it seem to be 5 additional input channels, 4 for the encoded masked-image and 1 for the mask itself. as per the sdxl model card

Edit2: NVM! I used the depth example and it seems to be related to 256 depth controlnet instead of instantid

skyler14 commented 2 months ago

inpainting is totally possible, the workflow is the same as standard inpainting. I'll post a workflow when I have time

any chance you might have some time