Mikubill / sd-webui-controlnet

WebUI extension for ControlNet
GNU General Public License v3.0
16.84k stars 1.94k forks source link

[Bug]: Inpaint produces completely white image #2762

Closed GravityGunner closed 5 months ago

GravityGunner commented 5 months ago

Is there an existing issue for this?

What happened?

Using the Inpaint Preprocessor no longer works, produces a blank image.

Steps to reproduce the problem

A very easy way to reproduce is to try to expand an image.

  1. In txt2img, Import an image into ControlNet.
  2. Select inpaint_only+lama
  3. Copy the image's dimensions to the Generation section's width and height sliders. Add a small amount to one of them - i.e. make the image wider (eg 512x768 to 768x768).
  4. Set Control Mode to 'ControlNet is more important' and Resize Mode to 'Resize and Fill'.
  5. Click Generate.

The image produced will be completely blank.

Also possible to reproduce under img2img by masking an area to Inpaint and using ControlNet + inpaint_only+lama as usual. Depending on the denoising strength this will not behave as expected.

What should have happened?

Inpainting fills in extra space as normal.

Commit where the problem happens

webui: adadb4e controlnet: 3a85645

Reverting to ControlNet commit a999981 resolves the problem.

What browsers do you use to access the UI ?

Apple Safari

Command Line Arguments

--xformers

List of enabled extensions

Config-Presets - https://github.com/Zyin055/Config-Presets.git
adetailer - https://github.com/Bing-su/adetailer.git sd-dynamic-prompts - https://github.com/adieyal/sd-dynamic-prompts.git

Console logs

2024-04-18 13:36:34,075 - ControlNet - INFO - unit_separate = False, style_align = False
2024-04-18 13:36:34,223 - ControlNet - INFO - Loading model: control_v11p_sd15_inpaint [ebff9138]
2024-04-18 13:36:34,478 - ControlNet - INFO - Loaded state_dict from [/home/alex/stable-diffusion-webui/models/ControlNet/control_v11p_sd15_inpaint.pth]
2024-04-18 13:36:34,478 - ControlNet - INFO - controlnet_default_config
2024-04-18 13:36:36,344 - ControlNet - INFO - ControlNet model control_v11p_sd15_inpaint [ebff9138](ControlModelType.ControlNet) loaded.
2024-04-18 13:36:36,344 - ControlNet - INFO - using mask
2024-04-18 13:36:36,378 - ControlNet - INFO - Using preprocessor: inpaint_only+lama
2024-04-18 13:36:36,378 - ControlNet - INFO - preprocessor resolution = 768
2024-04-18 13:36:37,284 - ControlNet - INFO - ControlNet used torch.float16 VAE to encode torch.Size([1, 4, 96, 96]).
2024-04-18 13:36:37,294 - ControlNet - INFO - ControlNet Hooked - Time = 3.234239101409912
2024-04-18 13:36:37,343 - ControlNet - INFO - [ControlNet] Initial noise hack applied to torch.Size([1, 4, 96, 96]).
100%|██████████████████████████████████████████████████████████████████| 20/20 [00:08<00:00,  2.42it/s]
Total progress: 100%|██████████████████████████████████████████████████| 20/20 [00:08<00:00,  2.40it/s]
2024-04-18 13:45:47,463 - ControlNet - INFO - unit_separate = False, style_align = False:00,  2.49it/s]
2024-04-18 13:45:47,669 - ControlNet - INFO - Loading model: control_v11p_sd15_inpaint [ebff9138]
2024-04-18 13:45:47,823 - ControlNet - INFO - Loaded state_dict from [/home/alex/stable-diffusion-webui/models/ControlNet/control_v11p_sd15_inpaint.pth]
2024-04-18 13:45:47,823 - ControlNet - INFO - controlnet_default_config
2024-04-18 13:45:49,640 - ControlNet - INFO - ControlNet model control_v11p_sd15_inpaint [ebff9138](ControlModelType.ControlNet) loaded.
2024-04-18 13:45:49,676 - ControlNet - INFO - Using preprocessor: inpaint_only+lama
2024-04-18 13:45:49,676 - ControlNet - INFO - preprocessor resolution = 512
2024-04-18 13:45:49,872 - ControlNet - INFO - ControlNet used torch.float16 VAE to encode torch.Size([1, 4, 64, 64]).
2024-04-18 13:45:49,876 - ControlNet - INFO - ControlNet Hooked - Time = 2.416152000427246
2024-04-18 13:45:50,025 - ControlNet - INFO - [ControlNet] Initial noise hack applied to torch.Size([1, 4, 64, 64]).

Additional information

No response

GravityGunner commented 5 months ago

Can confirm this has resolved the issue. Thanks for the quick response.