search

Acly / comfyui-inpaint-nodes

Nodes for better inpainting with ComfyUI: Fooocus inpaint model for SDXL, LaMa, MAT, and various other tools for pre-filling inpaint & outpaint areas.
GNU General Public License v3.0
577 stars 41 forks source link

"Soft Inpainting" feature from A1111 dev branch #12

Closed LuLmaster69 closed 6 months ago

LuLmaster69 commented 6 months ago

Thank you for greatly expanding the inpainting for ComfyUI.

It is not really an issue but I don't know how to raise this suggestion otherwise.

I was thinking that you may be interested in the new "soft inpainting" feature from the latest dev branch of Automatic1111. It seems that ComfyUI is currently lacking such a feature.

Here's more details on it: https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14208

Acly commented 6 months ago

I noticed that, and while I'm not 100% sure I think that is how denoise masks have always worked in ComfyUI. If you use only "VAE Encode (for Inpainting)" you get a binary mask. But "Set Latent Noise Mask" and relatively new "InpaintModelConditioning" use soft masks to add noise and for conditioning.

LuLmaster69 commented 6 months ago

Thanks for clarification!

nomadoor commented 6 months ago

InpaintModelConditioning InpaintModelConditioning.json I found it interesting and gave it a try, but it seems that neither InpaintModelConditioning nor Set Latent Noise Mask support non-binary masks. I apologize if I am mistaken.

LuLmaster69 commented 6 months ago

Reopening so that people could check if nomadoor's workflow is correct for non-binary masks.

Acly commented 6 months ago

You'd see a hard cut if it was rounded into a binary mask. Instead you see some kind of transition, but only for mask values close to 1, it seems to vanish quickly. Exactly how it uses mask values between 0 and 1 I don't know. The A1111 soft mask implements some kind of blending between denoised and original latent, probably this is different.

denoise-mask Comparison smooth vs binary

Acly commented 6 months ago

I think ComfyUI behavior works well enough for masks that are primarily fully opaque or transparent, but use soft edges / feathering at the borders for better transition. It's not suited if you need intuitive and arbitrary per-pixel denoise strength.

LuLmaster69 commented 6 months ago

Thanks, hopefully this would clarify things for people who may seek to implement per-pixel denoise inpainting in ComfyUI.

I also noticed that "soft inpainting" in dev Auto1111 with max blur changes the picture beyond the mask, as in the example provided in their pull request thread.

CodeHatchling commented 6 months ago

Thanks, hopefully this would clarify things for people who may seek to implement per-pixel denoise inpainting in ComfyUI.

I also noticed that "soft inpainting" in dev Auto1111 with max blur changes the picture beyond the mask, as in the example provided in their pull request thread.

Yes, in contrast to vanilla inpainting, any area within "blur radius" from a masked pixel may be modified. This is intended to help inpaint-sketch by expanding the influence of the denoiser to include all pixels that are affected by the sketch.