[Bug]: mask issue with inpaint if inpaint_only+lama selected

GeBo1 commented 1 year ago

What happened?

When I try to use openOutpaint with control_v11p_sd15_inpaint and inpaint_only+lama selected, it generates errors in the stable-diffusion-webui console window and the controlnet preprocessing is bypassed. If I run it without a mask it seems to execute without issue.

2023-09-06 12:09:34,155 - ControlNet - INFO - Loading model: control_v11p_sd15_inpaint [ebff9138]
2023-09-06 12:09:34,666 - ControlNet - INFO - Loaded state_dict from [W:\stable-diffusion-webui\webui\models\ControlNet\control_v11p_sd15_inpaint.pth]
2023-09-06 12:09:34,666 - ControlNet - INFO - controlnet_default_config
2023-09-06 12:09:37,784 - ControlNet - INFO - ControlNet model control_v11p_sd15_inpaint [ebff9138] loaded.
2023-09-06 12:09:37,928 - ControlNet - INFO - Loading preprocessor: inpaint_only+lama
2023-09-06 12:09:37,928 - ControlNet - INFO - preprocessor resolution = 64
*** Error running before_process_batch: W:\stable-diffusion-webui\webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "W:\stable-diffusion-webui\webui\modules\scripts.py", line 627, in before_process_batch
        script.before_process_batch(p, *script_args, **kwargs)
      File "W:\stable-diffusion-webui\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 972, in before_process_batch
        self.controlnet_main_entry(p)
      File "W:\stable-diffusion-webui\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 808, in controlnet_main_entry
        detected_map, is_image = preprocessor(
      File "W:\stable-diffusion-webui\webui\extensions\sd-webui-controlnet\scripts\utils.py", line 75, in decorated_func
        return cached_func(*args, **kwargs)
      File "W:\stable-diffusion-webui\webui\extensions\sd-webui-controlnet\scripts\utils.py", line 63, in cached_func
        return func(*args, **kwargs)
      File "W:\stable-diffusion-webui\webui\extensions\sd-webui-controlnet\scripts\global_state.py", line 35, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
      File "W:\stable-diffusion-webui\webui\extensions\sd-webui-controlnet\scripts\processor.py", line 489, in lama_inpaint
        prd_color = model_lama(img_res)
      File "W:\stable-diffusion-webui\webui\extensions\sd-webui-controlnet\annotator\lama\__init__.py", line 51, in __call__
        color = color * (1 - mask)
    RuntimeError: The size of tensor a (3) must match the size of tensor b (0) at non-singleton dimension 2

Steps to reproduce the problem

expand "Extensions"
check "ControlNet In/Outpainting"
Set "Inpaint Preprocessor" to inpaint_only+lama
Set "Model" to control_v11p_sd15_inpaint
Stable Diffusion settings, all boxes unchecked, resolution=512, steps=30, CFG Scale=5, Batch=2, Iterations=2, Mask blur=4
Draw a mask on image
Activate Img2img, select area that includes mask
check stable-diffusion-webui console window for errors

What should have happened?

Errors should not have occurred and things should proceed as expected

Commit where the problem happens

2f3c65e

What platforms do you use to access openOutpaint?

Windows

What browsers do you use to access the UI ?

Other (please list in additional information)

Browser Extensions/Addons

Vivaldi 6.2.3105.45 (blocking disabled for 127.0.01)

Script Safe (disabled for 127.0.0.1)
UBlock Origin (disabled for 127.0.0.1)
Privacy Badger (disabled for 127.0.0.1)
Tampermonkey (no scripts loaded for 127.0.0.1)

AUTOMATIC1111 webUI Commandline Arguments

--upcast-sampling --api --no-half-vae --xformers --medvram --cors-allow-origins *

Additional information

NOTE: inpaint_only+lama works fine using the standard inpaint interface

GeBo1 commented 1 year ago

I just realized it works if "Sync cursor size" is checked. Is it possible the mask image isn't being scaled? when Resolution != Cursor Size?

zero01101 commented 10 months ago

disclaimer: apologies for the extended delay, my brain is terrible and you shouldn't expect much from me

haven't gotten to reproduce this yet? will continue poking, it's highly likely something is indeed broken with extension support because it's extra jank as mentioned

zero01101 / openOutpaint