[Bug]: ControlNet shows controlnet is enabled but no input image is given in Txt2Img

Arron17 commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Have you updated WebUI and this extension to the newest version?

[X] I have updated WebUI and this extension to the most up-to-date version

Do you understand that you should go to https://github.com/IDEA-Research/Grounded-Segment-Anything/issues if you cannot install GroundingDINO?

[X] My problem is not about installing GroundingDINO

What happened?

When following the instructions for ControlNet Inpainting:

ControlNet Inpainting

Check Copy to ControlNet Inpaint and select the ControlNet panel for inpainting if you want to use multi-ControlNet. You can >be either at img2img tab or at txt2img tab to use this functionality. Configurate ControlNet panel. Click Enable, preprocessor choose inpaint_global_harmonious, model choose >control_v11p_sd15_inpaint [ebff9138]. There is no need to upload image to the ControlNet inpainting panel, as SAM extension will >help you to do that. Write your prompts, configurate A1111 panel and click Generate.

When using Txt2Img get a message to say there is no image input: Error running process: D:\Code\Stable-Diffusion\AUTOMATIC1111\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py Traceback (most recent call last): File "D:\Code\Stable-Diffusion\AUTOMATIC1111\stable-diffusion-webui\modules\scripts.py", line 417, in process script.process(p, *script_args) File "D:\Code\Stable-Diffusion\AUTOMATIC1111\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 779, in process raise ValueError('controlnet is enabled but no input image is given') ValueError: controlnet is enabled but no input image is given

If I follow the same steps in Img2Img a different error appears, however it does seem to actually generate the image correctly. Error in Img2Img is: Error running process: D:\Code\Stable-Diffusion\AUTOMATIC1111\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py Traceback (most recent call last): File "D:\Code\Stable-Diffusion\AUTOMATIC1111\stable-diffusion-webui\modules\scripts.py", line 417, in process script.process(p, *script_args) File "D:\Code\Stable-Diffusion\AUTOMATIC1111\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 808, in process detected_map, is_image = preprocessor(input_image, res=unit.processor_res, thr_a=unit.threshold_a, thr_b=unit.threshold_b) File "D:\Code\Stable-Diffusion\AUTOMATIC1111\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\processor.py", line 57, in inpaint mask = resize_image(img[:, :, 3:4], res) File "D:\Code\Stable-Diffusion\AUTOMATIC1111\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\util.py", line 33, in resize_image img = cv2.resize(input_image, (W, H), interpolation=cv2.INTER_LANCZOS4 if k > 1 else cv2.INTER_AREA) cv2.error: Unknown C++ exception from OpenCV code

Steps to reproduce the problem

Included images of my settings:

What should have happened?

The instructions state that I do not need to put an image into the ControlNet area as the extension will handle that. However it is giving an error saying there is no input image.

Commit where the problem happens

webui: 22bcc7be428c94e9408f589966c2040187245d81 extension: 4ee968b5a375e2c69e5769ed5c11bb18fa6e65f0

What browsers do you use to access the UI ?

No response

Command Line Arguments

--opt-sdp-attention --no-half-vae --opt-channelslast

Console logs

Included in the error message above

Additional information

No response

continue-revolution commented 1 year ago

You should update the ControlNet extension to the most updated version. I created a pull request to ControlNet extension several hours ago to support this functionality. For img2img, I will test and update here.

Edit: img2img works well on my side. You should update ControlNet extension.

Arron17 commented 1 year ago

You should update the ControlNet extension to the most updated version. I created a pull request to ControlNet extension several hours ago to support this functionality. For img2img, I will test and update here.

Edit: img2img works well on my side. You should update ControlNet extension.

I have no further updates for the ControlNet Extension, there was an update for Segment Anything which I updated. Issue is still present.

continue-revolution commented 1 year ago

please go to Extensions panel and check your controlnet version. if it is one of then it might be my problem. But I cannot reproduce your error. I think your UI setting is OK.

Arron17 commented 1 year ago

please go to Extensions panel and check your controlnet version. if it is one of then it might be my problem. But I cannot reproduce your error. I think your UI setting is OK.

sd-webui-controlnet https://github.com/Mikubill/sd-webui-controlnet 4cfa8aa7 (Mon Apr 17 20:04:26 2023) sd-webui-segment-anything https://github.com/continue-revolution/sd-webui-segment-anything.git 63d66389 (Mon Apr 17 22:32:41 2023)

Edit: I'll upload a video in a few minutes of what I'm doing, in case I'm doing something wrong.

continue-revolution commented 1 year ago

https://github.com/Mikubill/sd-webui-controlnet/issues/736#issuecomment-1512188769

I will try to reproduce your error on my end, but it's hard for me to understand the reason.

Arron17 commented 1 year ago

Mikubill/sd-webui-controlnet#736 (comment)

I will try to reproduce your error on my end, but it's hard for me to understand the reason.

Here is a video of what I'm doing for Txt2Img - https://streamable.com/s23pfe

continue-revolution commented 1 year ago

ok I understand. You should manually allow other scripts to control ControlNet extension. Some extension, such as posex, internally bypassed this requirement, but I don't think it is a good practice. Go to settings->controlnet->Allow other script to control this extension

Arron17 commented 1 year ago

ok I understand. You should manually allow other scripts to control ControlNet extension. Some extension, such as posex, internally bypassed this requirement, but I don't think it is a good practice. Go to settings->controlnet->Allow other script to control this extension

Ok, that has fixed both issues. Thank you.

It might be worth adding that into the ControlNet Instructions on the readme, as I wasn't aware that was an option.

continue-revolution commented 1 year ago

yes. I just updated my README and bug report checkbox to reflect this.

continue-revolution / sd-webui-segment-anything