search

lllyasviel / ControlNet-v1-1-nightly

Nightly release of ControlNet 1.1
4.48k stars 364 forks source link

[New Feature] Inpaint without changing unmasked area is implemented #57

Open lllyasviel opened 1 year ago

lllyasviel commented 1 year ago

After many reports about inpaint, we fixed the codes so that the gradio code becomes consistent to SOTA implementation of A1111 - it will not change unmasked areas anymore.

Before fix:

img

After fix:

img

You can see the color offsets on the cloth disappear. Now the unmasked area is 100% same with input images.

lllyasviel commented 1 year ago

Note that A1111 users can ignore this since A1111 controlnet already had this - see also https://github.com/Mikubill/sd-webui-controlnet/discussions/1143

Alexadar commented 1 year ago

Works fantastic thanks! How can I speed it up, besides xformers?

darshats commented 1 year ago

Hi, Thanks for the great work. I am curious about this line in the gradio inpaint.py line 89:

    x_samples = x_samples * mask_pixel_batched + img_pixel_batched * (1.0 - mask_pixel_batched)

If this line is not there, I observe that text in the unmasked area gets garbled. By overlaying the area outside the mask it is restored. So then this could have been done earlier implementation too. I'm missing something as the commit for this change does push the mask deeper into the sampler. Could you help understand the change, and why overlay is needed? thanks!