search

continue-revolution / sd-webui-animatediff

AnimateDiff for AUTOMATIC1111 Stable Diffusion WebUI
Other
3.09k stars 257 forks source link

[Bug]: ControlNet Hack Broken ? #363

Closed zopi4k closed 8 months ago

zopi4k commented 11 months ago

Is there an existing issue for this?

Have you read FAQ on README?

What happened?

When I want to activate control net, and adding my link path of my images folder, it gives me an error : TypeError: 'NoneType' object is not iterable. I have try the sames step without controlnet it's work, but when I acitvate control net : error

Steps to reproduce the problem

  1. chekpoint SD1.5
  2. Add Prompt
  3. Active the AnimateDiff with "enable" btn And change the model on "mm-Stabilized_mid.pth"
  4. Closed Loop "N" + 64 frames(number of pictures/frames to add in my folder) + 12 FPS
  5. I past the folder link in the Video Path
  6. I activate ControlNet with the model (control_v11f1p_sd15_depth) and no picture in the Single image input.
  7. Click Generate ...
  8. ... TypeError: 'NoneType' object is not iterable

What should have happened?

The Hack of AnimateDiff pass the pictures of my anime folder (0001.png to 0064.png) into the control net. But actualy is broken...

Commit where the problem happens

webui: 1.6.1 extension: • ControlNet v1.1.422 • AnimateDiff v1.12.1

What browsers do you use to access the UI ?

No response

Command Line Arguments

--no-half-vae --api --medvram

Console logs

no error in the browser log
2023-12-07 13:46:51,696 - AnimateDiff - INFO - AnimateDiff process start.
2023-12-07 13:46:51,696 - AnimateDiff - INFO - Motion module already injected. Trying to restore.
2023-12-07 13:46:51,697 - AnimateDiff - INFO - Restoring DDIM alpha.
2023-12-07 13:46:51,697 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet input blocks.
2023-12-07 13:46:51,697 - AnimateDiff - INFO - Removing motion module from SD1.5 UNet output blocks.
2023-12-07 13:46:51,697 - AnimateDiff - INFO - Restoring SD1.5 GroupNorm32 forward function.
2023-12-07 13:46:51,698 - AnimateDiff - INFO - Removal finished.
2023-12-07 13:46:51,707 - AnimateDiff - INFO - Hacking SD1.5 GroupNorm32 forward function.
2023-12-07 13:46:51,707 - AnimateDiff - INFO - Injecting motion module mm-Stabilized_mid.pth into SD1.5 UNet input blocks.
2023-12-07 13:46:51,707 - AnimateDiff - INFO - Injecting motion module mm-Stabilized_mid.pth into SD1.5 UNet output blocks.
2023-12-07 13:46:51,707 - AnimateDiff - INFO - Setting DDIM alpha.
2023-12-07 13:46:51,710 - AnimateDiff - INFO - Injection finished.
2023-12-07 13:46:51,710 - AnimateDiff - INFO - CFGDenoiser already hacked
2023-12-07 13:46:51,712 - AnimateDiff - INFO - Hacking ControlNet.
2023-12-07 13:46:51,712 - AnimateDiff - INFO - BatchHijack already hacked.
2023-12-07 13:46:51,712 - AnimateDiff - INFO - ControlNet Main Entry already hacked.
2023-12-07 13:46:51,715 - ControlNet - INFO - Loading model from cache: control_v11f1p_sd15_depth [cfd03158]
2023-12-07 13:46:52,775 - ControlNet - INFO - Loading preprocessor: none
2023-12-07 13:46:52,776 - ControlNet - INFO - preprocessor resolution = -1
100%|█████████████████████████████████████████████████████████████████████████████████| 64/64 [00:00<00:00, 171.34it/s]
*** Error running process: D:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "D:\stable-diffusion-webui\modules\scripts.py", line 619, in process
        script.process(p, *script_args)
      File "D:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1053, in process
        self.controlnet_hack(p)
      File "D:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1042, in controlnet_hack
        self.controlnet_main_entry(p)
      File "D:\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_cn.py", line 577, in hacked_main_entry
        self.latest_network.hook(model=unet, sd_ldm=sd_ldm, control_params=forward_params, process=p)
    TypeError: UnetHook.hook() missing 2 required positional arguments: 'batch_option_uint_separate' and 'batch_option_style_align'

---
  0%|                                                                                           | 0/12 [00:00<?, ?it/s]
*** Error completing request
*** Arguments: ('task(j9hpb8p7sdjw1cw)', 'treasure chest on a beach', '', ['(low quality:1.3)'], 12, 'LCM', 1, 1, 2, 512, 512, False, 0.6, 2, 'ESRGAN_4x', 10, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x00000238AB6F0790>, 0, False, '', 0.8, 1234, False, -1, 0, 0, 0, <scripts.animatediff_ui.AnimateDiffProcess object at 0x00000238AB6F0F10>, UiControlNetUnit(enabled=True, module='none', model='control_v11f1p_sd15_depth [cfd03158]', weight=0.5, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', save_detected_map=True), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=512, threshold_a=64, threshold_b=64, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', save_detected_map=True), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=512, threshold_a=64, threshold_b=64, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', save_detected_map=True), None, '', None, True, False, False, False, False, False, 0, 0, '0', 0, False, True, 0, 'Portrait of a [gender]', 'blurry', 20, ['DPM++ 2M Karras'], '', 0, None, 1, 1, '', False, False, False, 1, 0, 'Portrait of a [gender]', 'blurry', 20, ['DPM++ 2M Karras'], '', 0, None, '', None, True, False, False, False, False, False, 0, 0, '0', 0, False, True, 0, 'Portrait of a [gender]', 'blurry', 20, ['DPM++ 2M Karras'], '', 0, None, 1, 1, '', False, False, False, 1, 0, 'Portrait of a [gender]', 'blurry', 20, ['DPM++ 2M Karras'], '', 0, None, '', None, True, False, False, False, False, False, 0, 0, '0', 0, False, True, 0, 'Portrait of a [gender]', 'blurry', 20, ['DPM++ 2M Karras'], '', 0, None, 1, 1, '', False, False, False, 1, 0, 'Portrait of a [gender]', 'blurry', 20, ['DPM++ 2M Karras'], '', 0, None, 1, 1, '', 1, 1, ['After Upscaling/Before Restore Face'], 0, 'Portrait of a [gender]', 'blurry', 20, ['DPM++ 2M Karras'], '', 0, False, False, 'Matrix', 'Columns', 'Mask', 'Prompt', '1,1', '0.2', False, False, False, 'Attention', [False], '0', '0', '0.4', None, '0', '0', False, False, '1:1,1:2,1:2', '0:0,0:0,0:1', '0.2,0.8,0.8', 150, 0.2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50, [], 30, '', 4, [], 1, '', '', '', '') {}
    Traceback (most recent call last):
      File "D:\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "D:\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "D:\stable-diffusion-webui\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "D:\stable-diffusion-webui\modules\processing.py", line 732, in process_images
        res = process_images_inner(p)
      File "D:\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_cn.py", line 119, in hacked_processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "D:\stable-diffusion-webui\modules\processing.py", line 867, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "D:\stable-diffusion-webui\modules\processing.py", line 1140, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "D:\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 235, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "D:\stable-diffusion-webui\modules\sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "D:\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 235, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "D:\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_lcm.py", line 86, in sample_lcm
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "D:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "D:\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 256, in mm_cfg_forward
        x_out[a:b] = mm_sd_forward(self, x_in[a:b], sigma_in[a:b], subscript_cond(cond_in, a, b), subscript_cond(image_cond_in, a, b), make_condition_dict) # hook
      File "D:\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 159, in mm_sd_forward
        mm_cn_select(_context)
      File "D:\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_infv2v.py", line 109, in mm_cn_select
        for control in cn_script.latest_network.control_params:
    TypeError: 'NoneType' object is not iterable

---

Additional information

No response

oliverban commented 11 months ago

Yes, I was just trying to use CN (latest as of today) in A1111 and got the same error. Came here to report but you beat me to it. Guess it's not working anymore?

continue-revolution commented 11 months ago

A newer version of CN has probably something in conflict with AD. I am now also a dev in CN, as stated in #360 , which means that I will be able to address this when I am able to. However, as stated in #351 , I am still being trapped by a final fucking project in a very ridiculous course - I will not be able to do anything before I finish that. Please revert your CN back to commit https://github.com/Mikubill/sd-webui-controlnet/commit/10bd9b25f62deab9acb256301bbf3363c42645e7 via “git checkout \<commit hash>” in your CN folder.

continue-revolution commented 8 months ago

fixed