[Bug]: AttributeError: 'NoneType' object has no attribute 'convert'

[ongdien]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

I'm using 22bcc7be. In img2img I enable Controlnet and provide a png to the Controlnet canvas in order to use the Canny preprocessor. Clicking Generate, I got this error: AttributeError: 'NoneType' object has no attribute 'convert'

I fixed it by also provide a png to the img2img canvas. It seems to me the img2img canvas expects an image without knowing I'm using the Controlnet section for the generation.

Steps to reproduce the problem

1. Go to img2img (empty canvas)
2. Press Controlnet and copy an image to this canvas
3. Press generate. Error appears.

What should have happened?

No error and generate an image based on the canny preprocessor and the given prompts.

Commit where the problem happens

22bcc7b

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

@echo off

set PYTHON=
set GIT=
set VENV_DIR=
set COMMANDLINE_ARGS= 

call webui.bat  --theme=dark

List of extensions

Controlnet

Console logs

env "D:\STABLE DIFUSION\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
Commit hash: 22bcc7be428c94e9408f589966c2040187245d81
Fetching updates for Stable Diffusion...
Checking out commit for Stable Diffusion with hash: cf1d67a6fd5ea1aa600c4df58e5b47da45f6bdbf...
Installing requirements for Web UI

Launching Web UI with arguments: --theme=dark
No module 'xformers'. Proceeding without it.
Loading weights [d725be5d18] from D:\STABLE DIFUSION\stable-diffusion-webui\models\Stable-diffusion\revAnimated_v11.safetensors
Creating model from config: D:\STABLE DIFUSION\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading VAE weights specified in settings: D:\STABLE DIFUSION\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.ckpt
Applying cross attention optimization (Doggettx).
Textual inversion embeddings loaded(0):
Model loaded in 16.7s (load weights from disk: 0.3s, create model: 1.4s, apply weights to model: 1.3s, apply half(): 4.0s, load VAE: 0.6s, move model to device: 6.7s, load textual inversion embeddings: 2.4s).
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 27.6s (import torch: 2.6s, import gradio: 1.7s, import ldm: 1.0s, other imports: 1.8s, setup codeformer: 0.3s, load scripts: 2.2s, load SD checkpoint: 17.1s, create ui: 0.8s, gradio launch: 0.2s).
Error completing request
Arguments: ('task(x07g4u7c71ubb66)', 0, '', '', [], None, None, None, None, None, None, None, 20, 0, 4, 0, 1, False, False, 1, 1, 7, 1.5, 0.75, -1.0, -1.0, 0, 0, 0, False, 512, 512, 0, 0, 32, 0, '', '', '', [], 0, <scripts.external_code.ControlNetUnit object at 0x0000020E4AEC60E0>, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', 0, '', 0, '', True, False, False, False, 0, None, False, 50) {}
Traceback (most recent call last):
  File "D:\STABLE DIFUSION\stable-diffusion-webui\modules\call_queue.py", line 56, in f
    res = list(func(*args, **kwargs))
  File "D:\STABLE DIFUSION\stable-diffusion-webui\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "D:\STABLE DIFUSION\stable-diffusion-webui\modules\img2img.py", line 87, in img2img
    image = init_img.convert("RGB")
AttributeError: 'NoneType' object has no attribute 'convert'

Additional information

No response

[Z-nonymous]

@ongdien What you describe has to be done in txt2img tab. There, you don't use a source image and start from random noise (based on the seed).

img2img tabs is to take a source image as a starting point, If you don't specify an image there, then use txt2img instead.

You can see in the Arguments passed to img2img, it expects an image and you have None instead:

Arguments: ('task(x07g4u7c71ubb66)', 0, '', '', [], None, ...

def img2img(id_task: str, mode: int, prompt: str, negative_prompt: str, prompt_styles, init_img, ...

Maybe the UI should throw an error instead of calling img2img with None. But it any case the principle of img2img is to use a source image so there's nothing to be fixed here.

[ongdien]

Hi and thanks for the reply.Totally understood your reply.  However my question relates to an extension, Controlnet, one that's also available in the img2img tab, where users can provide an image to make a new image in the Controlnet section.  I didn't do this Controlnet operation on the txt2img tab because I thought I was doing an image to image operation. It's confusing because after searching online for possible causes, I provided an image to the img2img section to stop the error.  Which image was used then? I guess there can be some communication between an extension and the host for a smoother user experience. Thanks. On Thursday, April 6, 2023 at 04:23:30 AM PDT, Z-nonymous @.***> wrote:

@ongdien What you describe has to be done in txt2img tab. There, you don't use a source image and start from random noise (based on the seed).

img2img tabs is to take a source image as a starting point, If you don't specify an image there, then use txt2img instead.

You can see in the Arguments passed to img2img, it expects an image and you have None instead: Arguments: ('task(x07g4u7c71ubb66)', 0, '', '', [], None, ... def img2img(id_task: str, mode: int, prompt: str, negative_prompt: str, prompt_styles, init_img, ...

Maybe the UI should throw an error instead of calling img2img with None. But it any case the principle of img2img is to use a source image so there's nothing to be fixed here.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[Z-nonymous]

@ongdien I think it's a misunderstanding how this works.

ControlNet is a separate extension implementing this paper to help improve the results of a diffusion model (here Stable Diffusion). It adds additional controls to the diffusion to drive the output to a specific result. It works with both cases when using a prompt (txt2img) and when using a prompt+image (img2img).

When it's used in txt2img, You use a prompt and ControlNet as additional input to guide the results. When it's used in img2img, You use a prompt, a source image and ControlNet as additional input to guide the results.

ControlNet takes a specific input image as source (regardless of txt2img or img2img). It usually requires a specific type of image with a specific mode. For instance, the openpose model will need an openpose image, the depth model will need a depth map image, the canny model will need a canny edge image... So, you can input these images directly to ControlNet, or you can use the pre-processor on a regular image to create the input that will be suitable for a specific ControlNet model (canny, scribble, HED, pose, depthmap...).

All that works the same if using txt2img (just a prompt as input) or img2img (an image + prompt as input).

If you're wondering what is the difference between the two, with txt2img, it will start from random noise, and with img2img it will start from the image (not the one used by ControlNet) that it will denoise with a certain strength. Then, in both cases it will use the ControlNet image for specific control (i.e. an openpose, depth map, canny outline, scribble...) over the result. And then you can even chain multiple ControlNets (see Multi ControlNet) to further control the result.

You can make the test using same parameters (seed, steps, sampler, cfg scale...)

• txt2img, use a prompt + ControlNet with a specific image that you pre-process,
• img2img, same prompt + ControlNet with a specific image that you pre-process + same image as source for img2img,
• img2img, same prompt + ControlNet with a specific image that you pre-process + a different image as source for img2img,

Then you can see what effect of the source image is, when there is none (txt2img) or when there is one (img2img).

[Z-nonymous]

Here's an example:

Using canny model, same prompt ('Yoga pose') fixed seed, same steps/cfg scale/size... Image columns: First is the source image (No source image for txt2img) Second is image for ControlNet Third is result.

Just for the example, first line is with a ControlNet image that was not preprocessed for canny. Other lines use the same image but preprocessed for canny model using canny preprocessor. So that it is clear that ControlNet needs specific input (maybe it's more obvious using openpose or depth map)

Now you can see the differences between results in lines 2 to 4 which are due to a different starting point.

[Z-nonymous]

@ongdien Could you close this if resolved ?

[akx]

Please try updating your installation to the current master (1.3.0), and your controlnet extension too, and if this keeps happening, open a new issue.