Error occurred when using VAE Encode For Inpaint

comfyanonymous / ComfyUI

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.

https://www.comfy.org/

GNU General Public License v3.0

55.22k stars 5.84k forks source link

Error occurred when using VAE Encode For Inpaint #2535

Open cilantrocatsoup opened 9 months ago

cilantrocatsoup commented 9 months ago

I use KSamplerAdvanced for face replacement, generate a basic image with SDXL, and then use the 1.5 model to redraw the face with Refiner. Three results will emerge:

One is that the face can be replaced normally.
One is that the face is painted with a mask-like appearance.
The other is an error similar to the following: "output with shape [1, 1152, 896] doesn't match the broadcast shape [2, 1152, 896]."

ComfyUI_01947_ PixPin_2024-01-13_20-56-06 workflow1.json

nomadoor commented 9 months ago

inpaint inpaint.json

No errors occurred, but there were two mistakes that have been corrected:

VAEEncodeForInpaint → SetLatentNoiseMask
- VAEEncodeForInpaint fills the masked area with gray, so it can only be used when start_at_step is 0 and the inpainting model is used.
- (Even when using the inpainting model, it is now recommended to use InpaintModelConditioning)
Change add_noise of the second KSampler from disable to enable.

cilantrocatsoup commented 9 months ago

Thank you for your help. The error has been resolved. I feel that disable noise can better blend with the colors of the XL base image, so I did not enable it.

nomadoor commented 9 months ago

The "add_noise: disable" is used when processing sampling across two KSamplers. It doesn’t make much sense when there’s only one KSampler.

I feel that disable noise can better blend with the colors of the XL base image

However, this is not just your imagination. When sampling without adding noise, the details decrease and it becomes flat. It might be an effective technique to create a gentle atmosphere.