[Bug]: *** Error running process_before_every_sampling:

sumit91in commented 7 months ago

Checklist

[X] The issue exists after disabling all extensions
[X] The issue exists on a clean installation of webui
[X] The issue is caused by an extension, but I believe it is caused by a bug in the webui
[ ] The issue exists in the current version of the webui
[X] The issue has not been reported before recently
[X] The issue has been reported before but has not been fixed yet

What happened?

Facing issue with ControlNet in forge with SDXL models. For now tried segmentation. IP adaptors as well as instant ID in both img2img and text2img

Can anyone please help to help me with steps to resolve?

Steps to reproduce the problem

select any controlnet and click generate

What should have happened?

generation should have happened without issue.

What browsers do you use to access the UI ?

Microsoft Edge, Other

Sysinfo

.

Console logs

---
To load target model AutoencoderKL
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) =  2778.92431640625
[Memory Management] Model Memory (MB) =  319.11416244506836
[Memory Management] Minimal Inference Memory (MB) =  1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) =  1435.8101539611816
Moving model(s) has taken 0.09 seconds
*** Error running process_before_every_sampling: C:\Forge\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "C:\Forge\stable-diffusion-webui-forge\modules\scripts.py", line 835, in process_before_every_sampling
        script.process_before_every_sampling(p, *script_args, **kwargs)
      File "C:\Forge\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\Forge\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 555, in process_before_every_sampling
        self.process_unit_before_every_sampling(p, unit, self.current_params[i], *args, **kwargs)
    KeyError: 0

Additional information

No response

sumit91in commented 7 months ago

Traceback (most recent call last): File "C:\Forge\stable-diffusion-webui-forge\modules_forge\main_thread.py", line 37, in loop task.work() File "C:\Forge\stable-diffusion-webui-forge\modules_forge\main_thread.py", line 26, in work self.result = self.func(*self.args, self.kwargs) File "C:\Forge\stable-diffusion-webui-forge\modules\txt2img.py", line 111, in txt2img_function processed = processing.process_images(p) File "C:\Forge\stable-diffusion-webui-forge\modules\processing.py", line 752, in process_images res = process_images_inner(p) File "C:\Forge\stable-diffusion-webui-forge\modules\processing.py", line 922, in process_images_inner samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts) File "C:\Forge\stable-diffusion-webui-forge\modules\processing.py", line 1275, in sample samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x)) File "C:\Forge\stable-diffusion-webui-forge\modules\sd_samplers_kdiffusion.py", line 251, in sample samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, extra_params_kwargs)) File "C:\Forge\stable-diffusion-webui-forge\modules\sd_samplers_common.py", line 263, in launch_sampling return func() File "C:\Forge\stable-diffusion-webui-forge\modules\sd_samplers_kdiffusion.py", line 251, in samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, extra_params_kwargs)) File "C:\Forge\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "C:\Forge\stable-diffusion-webui-forge\repositories\k-diffusion\k_diffusion\sampling.py", line 145, in sample_euler_ancestral denoised = model(x, sigmas[i] s_in, extra_args) File "C:\Forge\stable-diffusion-webui-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "C:\Forge\stable-diffusion-webui-forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "C:\Forge\stable-diffusion-webui-forge\modules\sd_samplers_cfg_denoiser.py", line 182, in forward denoised = forge_sampler.forge_sample(self, denoiser_params=denoiser_params, File "C:\Forge\stable-diffusion-webui-forge\modules_forge\forge_sampler.py", line 88, in forge_sample denoised = sampling_function(model, x, timestep, uncond, cond, cond_scale, model_options, seed) File "C:\Forge\stable-diffusion-webui-forge\ldm_patched\modules\samplers.py", line 289, in sampling_function cond_pred, uncond_pred = calc_cond_uncondbatch(model, cond, uncond, x, timestep, model_options) File "C:\Forge\stable-diffusion-webui-forge\ldm_patched\modules\samplers.py", line 252, in calc_cond_uncond_batch c['control'] = control.get_control(inputx, timestep, control_cond, len(cond_or_uncond)) File "C:\Forge\stable-diffusion-webui-forge\ldm_patched\modules\controlnet.py", line 235, in get_control control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number) File "C:\Forge\stable-diffusion-webui-forge\ldm_patched\modules\controlnet.py", line 253, in get_control self.cond_hint = ldm_patched.modules.utils.common_upscale(self.cond_hint_original, x_noisy.shape[3] 8, x_noisy.shape[2] 8, 'nearest-exact', "center").to(dtype) File "C:\Forge\stable-diffusion-webui-forge\ldm_patched\modules\utils.py", line 394, in common_upscale old_width = samples.shape[3] AttributeError: 'dict' object has no attribute 'shape' 'dict' object has no attribute 'shape' Error completing request Arguments: ('task(8tr9gx06v7e8dk3)', <gradio.routes.Request object at 0x0000026F53A0A680>, ', short hair, brown eyes, pale skin girl, 45 years pale skin, freckles, turtleneck, bare shoulders, blush, red rose in hair, looking_longingly, seductive, sexy_woman, black background', 'score_6, score_5, score_4, pony, gaping, muscular, censored, furry, child, kid, chibi, monochrome', [], 30, 'Euler a', 3, 1, 7, 1024, 720, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], 0, False, '', 0.8, -1, False, -1, 0, 0, 0, True, False, False, False, 'base', False, False, {'ad_model': 'mediapipe_face_full', 'ad_model_classes': '', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_model_classes': '', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, None, False, '0', '0', 'inswapper_128.onnx', 'CodeFormer', 1, True, 'None', 1, 1, False, True, 1, 0, 0, False, 0.5, True, False, 'CUDA', False, 0, 'None', '', None, False, False, 0.5, 0, ControlNetUnit(input_mode=<InputMode.SIMPLE: 'simple'>, use_preview_as_input=False, batch_image_dir='', batch_mask_dir='', batch_input_gallery=[], batch_mask_gallery=[], generated_image=None, mask_image=None, hr_option='Both', enabled=True, module='InsightFace (InstantID)', model='control_instant_id_sdxl [c5c25a50]', weight=1, image={'image': array([[[133, 135, 134], [134, 136, 135], [135, 137, 136], ..., [189, 194, 198], [201, 206, 210], *** [ 0, 0, 0]],

[[133, 135, 134], [133, 135, 134], [135, 137, 136], ..., [189, 193, 198], [201, 206, 210], *** [ 0, 0, 0]],

[[130, 132, 131], [133, 135, 134], [135, 137, 136], ..., [187, 197, 200], [201, 209, 212], *** [ 0, 0, 0]],

*** ...,

[[139, 105, 61], [126, 89, 42], [152, 121, 74], ..., [ 5, 7, 6], [ 0, 0, 0], *** [ 14, 14, 14]],

[[148, 112, 66], [155, 124, 76], [160, 129, 82], ..., [ 3, 5, 4], [ 4, 7, 5], *** [ 15, 15, 15]],

[[151, 121, 73], [151, 119, 72], [156, 124, 77], ..., [ 5, 7, 6], [ 1, 5, 4], [ 15, 15, 15]]], dtype=uint8), 'mask': array([[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], *** [0, 0, 0]],

[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], *** [0, 0, 0]],

*** ...,

[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], *** [0, 0, 0]],

[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], [0, 0, 0]]], dtype=uint8)}, resize_mode='Crop and Resize', processor_res=0.5, threshold_a=0.5, threshold_b=0.5, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='ControlNet is more important', save_detected_map=True), ControlNetUnit(input_mode=<InputMode.SIMPLE: 'simple'>, use_preview_as_input=False, batch_image_dir='', batch_mask_dir='', batch_input_gallery=[], batch_mask_gallery=[], generated_image=None, mask_image=None, hr_option='Both', enabled=True, module='InsightFace (InstantID)', model='control_instant_id_sdxl [c5c25a50]', weight=1, image={'image': array([[[ 0, 0, 0], [ 0, 0, 0], [ 0, 0, 0], ..., [ 0, 0, 0], [ 0, 0, 0], *** [ 0, 0, 0]],

[[ 0, 0, 0], [ 0, 0, 0], [ 0, 0, 0], ..., [ 0, 0, 0], [ 0, 0, 0], *** [ 0, 0, 0]],

*** ...,

[[ 0, 0, 0], [186, 117, 67], [190, 121, 71], ..., [ 0, 0, 0], [ 0, 0, 0], *** [ 0, 0, 0]],

[[ 0, 0, 0], [181, 113, 64], [186, 118, 70], ..., [ 0, 0, 0], [ 0, 0, 0], *** [ 0, 0, 0]],

[[ 0, 0, 0], [176, 110, 60], [183, 117, 66], ..., [ 0, 0, 0], [ 0, 0, 0], [ 0, 0, 0]]], dtype=uint8), 'mask': array([[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], *** [0, 0, 0]],

[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], *** [0, 0, 0]],

*** ...,

[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], *** [0, 0, 0]],

[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], [0, 0, 0]]], dtype=uint8)}, resize_mode='Crop and Resize', processor_res=0.5, threshold_a=0.5, threshold_b=0.5, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='ControlNet is more important', save_detected_map=True), False, 7, 1, 'Constant', 0, 'Constant', 0, 1, 'enable', 'MEAN', 'AD', 1, False, 1.01, 1.02, 0.99, 0.95, False, 0.5, 2, False, 256, 2, 0, False, False, 3, 2, 0, 0.35, True, 'bicubic', 'bicubic', False, 0, 'anisotropic', 0, 'reinhard', 100, 0, 'subtract', 0, 0, 'gaussian', 'add', 0, 100, 127, 0, 'hard_clamp', 5, 0, 'None', 'None', False, 'MultiDiffusion', 768, 768, 64, 4, False, False, False, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, False, False, False, 0, False) {} Traceback (most recent call last): File "C:\Forge\stable-diffusion-webui-forge\modules\call_queue.py", line 57, in f res = list(func(args, kwargs)) TypeError: 'NoneType' object is not iterable