[Bug]: AttributeError: 'NoneType' object has no attribute 'mode'

FournyP commented 9 months ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

I got a error when trying to generate :

*** Error completing request
*** Arguments: ('task(noa5c2sel95lyex)', 0, "Two characters. A woman with dark hair pulled back, wearing a sleeveless black top. A man with short dark hair, dressed in a dark suit. It's an indoor setting with a white door in the background. The lighting appears to be bright, likely from artificial sources given the indoor setting.", 'deformed, disfigured, ugly', [], None, None, None, None, None, None, None, 20, 'DPM++ 2M Karras', 4, 0, 1, 1, 1, 7, 1.5, 0.75, 0, 512, 512, 1, 0, 0, 32, 0, '', '', '', [], False, [], '', <gradio.routes.Request object at 0x00000225AB533E50>, 9, False, '', 0.8, -1, False, -1, 0, 0, 0, UiControlNetUnit(enabled=True, module='lineart_anime', model='control_v11p_sd15_normalbae [316696f1]', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=512, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None), '* `CFG Scale` should be 2 or lower.', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, 'start', '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, 'C:\\Users\\rozuk\\AppData\\Local\\Temp\\2\\gradio\\720dfdcd3b5dd64e69d083da0b07425dc456cec4\\yvbgto.mp4', None, False, None, None, False, None, None, False, 50) {}
    Traceback (most recent call last):
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\img2img.py", line 236, in img2img
        processed = modules.scripts.scripts_img2img.run(p, *args)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\scripts.py", line 700, in run
        processed = script.run(p, *script_args)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\movie2movie.py", line 146, in run
        proc = process_images(copy_p)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\processing.py", line 734, in process_images
        res = process_images_inner(p)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\processing.py", line 804, in process_images_inner
        p.init(p.all_prompts, p.all_seeds, p.all_subseeds)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\processing.py", line 1447, in init
        image = images.flatten(img, opts.img2img_background_color)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\images.py", line 788, in flatten
        if img.mode == "RGBA":
    AttributeError: 'NoneType' object has no attribute 'mode'

I run a Google Cloud instance. I've try on debian machine with --listen flag and by reaching the service through my browser. I also try on a Windows Server without --listen flag, directly on localhost using rdp but I got the exact same error. Both with a L4 Nvidia GPU.

I've also try different videos

Steps to reproduce the problem

Install Stable Diffusion web ui using this tutoriel : https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-NVidia-GPUs#windows-method-2
Install some checkpoints
Install sd-webui-controlnet like this :
Return to installed tab and click on Check for updates and then Apply and restart UI
Reboot the system, or close the terminal (in my case I've reboot the system).
Download models on HuggingFace (https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main)
Put a video, and options like this :
Got the error :

What should have happened?

A image generation I guess

Commit where the problem happens

webui: cf2772fab0af5573da775e7437e6acdca424f26e (last one in main branch) controlnet: 9d1f0a07fa2754338a6b260fad2abaf9d3f42ade (last one in main branch)

What browsers do you use to access the UI ?

Brave, Microsoft Edge

Command Line Arguments

none on `Windows Server`
`--listen` on Debian

List of enabled extensions

Console logs

*** Error completing request
*** Arguments: ('task(noa5c2sel95lyex)', 0, "Two characters. A woman with dark hair pulled back, wearing a sleeveless black top. A man with short dark hair, dressed in a dark suit. It's an indoor setting with a white door in the background. The lighting appears to be bright, likely from artificial sources given the indoor setting.", 'deformed, disfigured, ugly', [], None, None, None, None, None, None, None, 20, 'DPM++ 2M Karras', 4, 0, 1, 1, 1, 7, 1.5, 0.75, 0, 512, 512, 1, 0, 0, 32, 0, '', '', '', [], False, [], '', <gradio.routes.Request object at 0x00000225AB533E50>, 9, False, '', 0.8, -1, False, -1, 0, 0, 0, UiControlNetUnit(enabled=True, module='lineart_anime', model='control_v11p_sd15_normalbae [316696f1]', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=512, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None), '* `CFG Scale` should be 2 or lower.', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, 'start', '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, 'C:\\Users\\rozuk\\AppData\\Local\\Temp\\2\\gradio\\720dfdcd3b5dd64e69d083da0b07425dc456cec4\\yvbgto.mp4', None, False, None, None, False, None, None, False, 50) {}
    Traceback (most recent call last):
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\img2img.py", line 236, in img2img
        processed = modules.scripts.scripts_img2img.run(p, *args)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\scripts.py", line 700, in run
        processed = script.run(p, *script_args)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\movie2movie.py", line 146, in run
        proc = process_images(copy_p)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\processing.py", line 734, in process_images
        res = process_images_inner(p)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\processing.py", line 804, in process_images_inner
        p.init(p.all_prompts, p.all_seeds, p.all_subseeds)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\processing.py", line 1447, in init
        image = images.flatten(img, opts.img2img_background_color)
      File "C:\Users\rozuk\Desktop\stable-diffusion-webui\modules\images.py", line 788, in flatten
        if img.mode == "RGBA":
    AttributeError: 'NoneType' object has no attribute 'mode'

Additional information

No response

huchenlei commented 9 months ago

What do you mean by

Put a video, and options like this

I don't think A1111 handles video2video.

FournyP commented 9 months ago

Hi @huchenlei,

sd-webui-controlnet allows you to input "Movie", as shown in this video: https://youtu.be/6FCbcHm7vEc?si=nMpgzdsZam0sRgwT&t=200.

Perhaps I've misunderstood something, but why provide the option to input video if it's not compatible?

EDIT: I just realized from the video that the person used txt2img instead of img2img. I initially thought img2img was the obvious choice for videos, considering they are sequences of images. My bad. I'll try using txt2img instead.

huchenlei commented 9 months ago

The person in the video selects "ControlNet M2M" in script. If you don't do that ControlNet won't accept video.

Mikubill / sd-webui-controlnet