search

pkuliyi2015 / multidiffusion-upscaler-for-automatic1111

Tiled Diffusion and VAE optimize, licensed under CC BY-NC-SA 4.0
Other
4.75k stars 334 forks source link

KeyError: 0 when using Noise Inversion with Reigonal Prompting and Tiled Diffusion #136

Open mariaWitch opened 1 year ago

mariaWitch commented 1 year ago

Currently multidiffusion errors out when using any sampler if Noise Inversion and Regional Prompting are enabled together. Below are my settings for tiled diffusion.

image

And this is the backtrace:

Arguments: ('task(zfstmq4f3rlelzl)', 0, 'masterpiece, best quality, highres, extremely detailed 8k wallpaper, very clear', '(worst quality, low quality:1.4), (Dappling, Pointillism :1.4), blur, blurry, depth of field, monochrome, by (bad-artist:0.6), EasyNegative', [], <PIL.Image.Image image mode=RGBA size=1440x1152 at 0x7F13FFB7DD50>, None, None, None, None, None, None, 20, 19, 4, 0, 1, False, False, 1, 1, 11, 1.5, 0.35, 274876147.0, -1.0, 0, 0, 0, False, 1152, 1440, 0, 0, 32, 0, '', '', '', [], 0, True, 'Mixture of Diffusers', True, 10, 1, 1, 64, False, True, 1024, 1024, 128, 128, 16, 8, 'sudo_RealESRGAN2x_3.332.758_G', 2, False, True, False, False, True, 0.5000333333333333, 0.1417, 0.20669999999999997, 0.2427, 'houshou marine, red hair, heterochromia, red eye ,yellow eye, red hair,', '', 'Foreground', 0.2, 274876147.0, True, 0.6067, 0.7708, 0.1333, 0.2042, 'ice skate', '', 'Foreground', 0.2, -1.0, True, 0, 0, 1, 1, 'hockey stadium, ice hockey, ice hockey rink, hockey rink,', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, True, False, True, True, False, 1536, 96, False, '', 0, False, 7, 100, 'Constant', 0, 'Constant', 0, 4, False, 1, 0.15, False, 'OUT', ['OUT'], 5, 0, 'Bilinear', False, 'Pooling Max', False, 'Lerp', '', '', False, False, False, 'Horizontal', '1,1', '0.2', False, False, False, 'Attention', False, False, False, 1, False, False, 3, 0, False, False, 0, False, False, False, False, False, '1:1,1:2,1:2', '0:0,0:0,0:1', '0.2,0.8,0.8', 20, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, False, 1, '', 0, '', 0, '', True, False, False, False, 0, 'Random', False, False, 1, 'None', 'None', 'None', '<p style="margin-bottom:0.75em">Will upscale the image depending on the selected target size type</p>', 512, 0, 8, 32, 64, 0.35, 32, 0, True, 0, False, 8, 0, 0, 2048, 2048, 2) {}
Traceback (most recent call last):
  File "/home/maria/stable-diffusion-webui/modules/call_queue.py", line 56, in f
    res = list(func(*args, **kwargs))
  File "/home/maria/stable-diffusion-webui/modules/call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "/home/maria/stable-diffusion-webui/modules/img2img.py", line 172, in img2img
    processed = process_images(p)
  File "/home/maria/stable-diffusion-webui/modules/processing.py", line 508, in process_images
    res = process_images_inner(p)
  File "/home/maria/stable-diffusion-webui/modules/processing.py", line 663, in process_images_inner
    samples_ddim = p.sample(conditioning=c, unconditional_conditioning=uc, seeds=seeds, subseeds=subseeds, subseed_strength=p.subseed_strength, prompts=prompts)
  File "/home/maria/stable-diffusion-webui/modules/processing.py", line 1109, in sample
    samples = self.sampler.sample_img2img(self, self.init_latent, x, conditioning, unconditional_conditioning, image_conditioning=self.image_conditioning)
  File "/home/maria/stable-diffusion-webui/extensions/multidiffusion-upscaler-for-automatic1111/tile_utils/utils.py", line 249, in wrapper
    return fn(*args, **kwargs)
  File "/home/maria/stable-diffusion-webui/extensions/multidiffusion-upscaler-for-automatic1111/tile_methods/abstractdiffusion.py", line 532, in sample_img2img
    latent = self.find_noise_for_image_sigma_adjustment(sampler.model_wrap, self.noise_inverse_steps, prompts)
  File "/home/maria/stable-diffusion-webui/venvLinux/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/maria/stable-diffusion-webui/extensions/multidiffusion-upscaler-for-automatic1111/tile_methods/abstractdiffusion.py", line 624, in find_noise_for_image_sigma_adjustment
    eps = self.get_noise(x_in * c_in, t, cond_in, steps - i)
  File "/home/maria/stable-diffusion-webui/venvLinux/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/maria/stable-diffusion-webui/extensions/multidiffusion-upscaler-for-automatic1111/tile_methods/mixtureofdiffusers.py", line 193, in get_noise
    return self.apply_model_hijack(x_in, sigma_in, cond=cond_in, noise_inverse_step=step)
  File "/home/maria/stable-diffusion-webui/venvLinux/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/maria/stable-diffusion-webui/extensions/multidiffusion-upscaler-for-automatic1111/tile_utils/utils.py", line 249, in wrapper
    return fn(*args, **kwargs)
  File "/home/maria/stable-diffusion-webui/extensions/multidiffusion-upscaler-for-automatic1111/tile_methods/mixtureofdiffusers.py", line 159, in apply_model_hijack
    x_tile_out = shared.sd_model.apply_model(x_tile, t_in, cond=custom_cond_in)
  File "/home/maria/stable-diffusion-webui/venvLinux/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/maria/stable-diffusion-webui/extensions/multidiffusion-upscaler-for-automatic1111/tile_utils/utils.py", line 249, in wrapper
    return fn(*args, **kwargs)
  File "/home/maria/stable-diffusion-webui/extensions/multidiffusion-upscaler-for-automatic1111/tile_methods/mixtureofdiffusers.py", line 151, in apply_model_hijack
    x_tile_out = self.custom_apply_model(x_tile, t_in, c_in, bbox_id, bbox)
  File "/home/maria/stable-diffusion-webui/extensions/multidiffusion-upscaler-for-automatic1111/tile_methods/mixtureofdiffusers.py", line 69, in custom_apply_model
    return self.kdiff_custom_forward(x_in, t_in, c_in, bbox_id, bbox, forward_func=shared.sd_model.apply_model_original_md)
  File "/home/maria/stable-diffusion-webui/extensions/multidiffusion-upscaler-for-automatic1111/tile_utils/utils.py", line 249, in wrapper
    return fn(*args, **kwargs)
  File "/home/maria/stable-diffusion-webui/extensions/multidiffusion-upscaler-for-automatic1111/tile_methods/abstractdiffusion.py", line 263, in kdiff_custom_forward
    tensor = self.tensor[bbox_id]
KeyError: 0
dpinav commented 1 year ago

I think this is still a problem. Seems to work if you use Multidiffusion instead of Mixture of Diffusers though.