Auto-Photoshop-StableDiffusion-Plugin control net not working anymore

phinck commented 3 weeks ago

I am encountering an issue with a Photoshop plugin for Automatic1111 that I use to enhance my sketches with the lineart preprocessor. Additionally, I frequently use depth and canny preprocessors, which have significantly improved my productivity. ControlNet functions well within the WEBUI but fails when used in Photoshop.

I've attempted rolling back my version of AUTO1111 without success. Troubleshooting output errors using Python GPT in ChatGPT-4 has also not revealed the root cause. I hope to find a resolution here.

Here are the specific warnings and errors I'm encountering:

ControlNet - WARNING - Deprecated alias 'input_image' detected. This field will be removed on 2024-06-01Please use 'image' instead ControlNet - WARNING - Deprecated alias 'lowvram' detected. This field will be removed on 2024-06-01Please use 'low_vram' instead.

I edited the controlnet.py script in Notepad++, replacing all instances of 'input_image' with 'image' and 'lowvram' with 'low_vram'. This resolved the warnings, but the main issue with the preprocessors in the Photoshop plugin persists.

Here is the output while trying to use the lineart preprocessor, which also occurs with all ControlNet preprocessors:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Loading A111 WebUI Launcher ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ i Settings file found, loading → Updating Settings File ✓ i Launcher Version 1.7.0 i Found a custom WebUI Config i No Launcher launch options ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ → Checking requirements : i Python 3.10.6150.1013 found in registry: C:\Users\Parker\AppData\Local\Programs\Python\Python310\ i Clearing PATH of any mention of Python → Adding python 3.10 to path ✓ i Git found and already in PATH: C:\Program Files\Git\cmd\git.exe i Automatic1111 SD WebUI found: F:\AI\AUTO1111\stable-diffusion-webui i One or more checkpoint models were found Get-Content : Cannot find path 'F:\AI\AUTO1111\stable-diffusion-webui.git\refs\heads\master' because it does not exist. At F:\AI\AUTO1111\Launcher\Functions.psm1:167 char:13

$hash = Get-Content $hashPath


+ CategoryInfo          : ObjectNotFound: (F:\AI\AUTO1111\...fs\heads\master:String) [Get-Content], ItemNotFoundEx
ception
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Loading Complete, opening launcher ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ i Arguments are now: --xformers --ckpt-dir 'F:\AI\AUTO1111\models' --api ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ↺ Updating Extension: Auto-Photoshop-StableDiffusion-Plugin ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ↺ Updating Extension: sd-webui-controlnet From https://github.com/Mikubill/sd-webui-controlnet c91dbe5..bb3c2a9 main -> origin/main ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ↺ Updating Extension: sd-webui-reactor ✓ Done i Arguments are now: --xformers --ckpt-dir 'F:\AI\AUTO1111\models' --api ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ WEBUI LAUNCHING VIA EMS LAUNCHER, EXIT THIS WINDOW TO STOP THE WEBUI ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ! Any error happening after 'commit hash : XXXX' is not related to the launcher. Please report them on Automatic1111's github instead : ☁ https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/new/choose ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Cancel venv "F:\AI\AUTO1111\stable-diffusion-webui\venv\Scripts\Python.exe" fatal: not a git repository (or any of the parent directories): .git fatal: not a git repository (or any of the parent directories): .git Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] Version: 1.8.0-RC Commit hash: Installing requirements [Auto-Photoshop-SD] Attempting auto-update... [Auto-Photoshop-SD] switch branch to extension branch. checkout_result: Your branch is up to date with 'origin/master'.

[Auto-Photoshop-SD] Current Branch. branch_result: * master

[Auto-Photoshop-SD] Fetch upstream. fetch_result: [Auto-Photoshop-SD] Pull upstream. pull_result: Already up to date. Installing sd-webui-controlnet requirement: changing opencv-python version from 4.10.0.82 to 4.8.0 CUDA 12.1 Launching Web UI with arguments: --autolaunch --xformers --ckpt-dir F:\AI\AUTO1111\models --api python_server_full_path: F:\AI\AUTO1111\stable-diffusion-webui\extensions\Auto-Photoshop-StableDiffusion-Plugin\server/python_server ControlNet preprocessor location: F:\AI\AUTO1111\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads 2024-06-05 16:39:36,666 - ControlNet - INFO - ControlNet v1.1.449 16:39:37 - ReActor - STATUS - Running v0.7.0-b7 on Device: CUDA Loading weights [656abf9872] from F:\AI\AUTO1111\stable-diffusion-webui\models\Stable-diffusion\3dCartoonVision_v10.safetensors 2024-06-05 16:39:37,513 - ControlNet - INFO - ControlNet UI callback registered. Creating model from config: F:\AI\AUTO1111\stable-diffusion-webui\configs\v1-inference.yaml Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). Startup time: 28.0s (prepare environment: 16.4s, import torch: 4.9s, import gradio: 1.0s, setup paths: 1.1s, initialize shared: 0.3s, other imports: 0.6s, load scripts: 2.2s, create ui: 0.7s, gradio launch: 0.4s, add APIs: 0.4s). Applying attention optimization: xformers... done. Model loaded in 3.7s (load weights from disk: 0.3s, create model: 1.2s, apply weights to model: 1.9s, calculate empty prompt: 0.1s). 2024-06-05 16:40:57,223 - ControlNet - WARNING - Deprecated alias 'lowvram' detected. This field will be removed on 2024-06-01Please use 'low_vram' instead. 2024-06-05 16:40:57,223 - ControlNet - WARNING - Deprecated alias 'input_image' detected. This field will be removed on 2024-06-01Please use 'image' instead. 2024-06-05 16:40:57,224 - ControlNet - INFO - [invert (from white bg & black line).processor_res] Invalid value(512), using default value 0.5. 2024-06-05 16:40:57,225 - ControlNet - WARNING - Deprecated alias 'lowvram' detected. This field will be removed on 2024-06-01Please use 'low_vram' instead. 2024-06-05 16:40:57,225 - ControlNet - WARNING - Deprecated alias 'input_image' detected. This field will be removed on 2024-06-01Please use 'image' instead. 2024-06-05 16:40:57,226 - ControlNet - WARNING - Deprecated alias 'lowvram' detected. This field will be removed on 2024-06-01Please use 'low_vram' instead. 2024-06-05 16:40:57,226 - ControlNet - WARNING - Deprecated alias 'input_image' detected. This field will be removed on 2024-06-01Please use 'image' instead. 2024-06-05 16:40:57,230 - ControlNet - WARNING - Deprecated alias 'lowvram' detected. This field will be removed on 2024-06-01Please use 'low_vram' instead. 2024-06-05 16:40:57,230 - ControlNet - WARNING - Deprecated alias 'input_image' detected. This field will be removed on 2024-06-01Please use 'image' instead. 2024-06-05 16:40:57,231 - ControlNet - INFO - [invert (from white bg & black line).processor_res] Invalid value(512), using default value 0.5. 2024-06-05 16:40:57,231 - ControlNet - WARNING - Deprecated alias 'lowvram' detected. This field will be removed on 2024-06-01Please use 'low_vram' instead. 2024-06-05 16:40:57,232 - ControlNet - WARNING - Deprecated alias 'input_image' detected. This field will be removed on 2024-06-01Please use 'image' instead. 2024-06-05 16:40:57,233 - ControlNet - WARNING - Deprecated alias 'lowvram' detected. This field will be removed on 2024-06-01Please use 'low_vram' instead. 2024-06-05 16:40:57,234 - ControlNet - WARNING - Deprecated alias 'input_image' detected. This field will be removed on 2024-06-01Please use 'image' instead. 2024-06-05 16:40:57,235 - ControlNet - INFO - unit_separate = False, style_align = False 2024-06-05 16:40:57,417 - ControlNet - INFO - Loading model: control_v11p_sd15_lineart [43d4be0d] 2024-06-05 16:40:57,956 - ControlNet - INFO - Loaded state_dict from [F:\AI\AUTO1111\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11p_sd15_lineart.pth] 2024-06-05 16:40:57,957 - ControlNet - INFO - controlnet_default_config 2024-06-05 16:41:00,043 - ControlNet - INFO - ControlNet model control_v11p_sd15_lineart 43d4be0d loaded. *** Error running process: F:\AI\AUTO1111\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py Traceback (most recent call last): File "F:\AI\AUTO1111\stable-diffusion-webui\modules\api\api.py", line 96, in decode_base64_to_image image = Image.open(BytesIO(base64.b64decode(encoding))) File "F:\AI\AUTO1111\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 3298, in open raise UnidentifiedImageError(msg) PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x00000212212CFBF0>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "F:\AI\AUTO1111\stable-diffusion-webui\modules\scripts.py", line 784, in process
    script.process(p, *script_args)
  File "F:\AI\AUTO1111\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1222, in process
    self.controlnet_hack(p)
  File "F:\AI\AUTO1111\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1207, in controlnet_hack
    self.controlnet_main_entry(p)
  File "F:\AI\AUTO1111\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 941, in controlnet_main_entry
    controls, hr_controls, additional_maps = get_control(
  File "F:\AI\AUTO1111\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 206, in get_control
    input_image, resize_mode = Script.choose_input_image(p, unit, idx)
  File "F:\AI\AUTO1111\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 692, in choose_input_image
    image = unit.get_input_images_rgba()
  File "F:\AI\AUTO1111\stable-diffusion-webui\extensions\sd-webui-controlnet\internal_controlnet\args.py", line 407, in get_input_images_rgba
    np_mask = self.parse_image(mask) if mask is not None else None
  File "F:\AI\AUTO1111\stable-diffusion-webui\extensions\sd-webui-controlnet\internal_controlnet\args.py", line 284, in parse_image
    np_image = cls.cls_decode_base64(image)
  File "F:\AI\AUTO1111\stable-diffusion-webui\extensions\sd-webui-controlnet\internal_controlnet\external_code.py", line 123, in to_base64_nparray
    return np.array(api.decode_base64_to_image(encoding)).astype("uint8")
  File "F:\AI\AUTO1111\stable-diffusion-webui\modules\api\api.py", line 99, in decode_base64_to_image
    raise HTTPException(status_code=500, detail="Invalid encoded image") from e
fastapi.exceptions.HTTPException

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:04<00:00, 4.25it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:04<00:00, 4.60it/s]

Thank you for all your help in advance.

huchenlei commented 3 weeks ago

Actually I forgot to deprecate these aliases. So they are still working and not the issue.

huchenlei commented 3 weeks ago

I think the issue is that we are trying to parse a mask image and expects a base64 image from the API, but somehow the string is not a base64 image string.

Can you file an issue in the SD photoshop extension repo and reference this issue there?

phinck commented 3 weeks ago

Yes, I will. Thank you for the insight into this.

huchenlei commented 3 weeks ago

It is probably broken by refactor I did in 1.1.448 on ControlNetUnit creation. You can switch to 1.1.447 and it should be working for now.

You can do the switch by git co 1b95e476ecf04d1abe652a61ff2e782628173aa7 in sd-webui-controlnet repo to the last commit before the breaking change PR.

phinck commented 3 weeks ago

would you clarify as where I should clone this to? Is it fine to direct the clone to the main directory? VENV folder?

huchenlei commented 3 weeks ago

Sorry about that. I mean git checkout instead of git clone here.

phinck commented 3 weeks ago

thank you. to be clear, I should execute this command in the sd-webui-controlnet folder within the extensions main folder?

huchenlei commented 3 weeks ago

in the sd-webui-controlnet folder

phinck commented 3 weeks ago

it worked great! Thank you so much!

Mikubill / sd-webui-controlnet

Auto-Photoshop-StableDiffusion-Plugin control net not working anymore #2948