[Bug]: IndexError: index 1 is out of bounds for dimension 0 with size 1 when Soft inpainting enabled and batched img2img

[wildwindfeng]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Have you read FAQ on README?

• [X] I have updated WebUI and this extension to the latest version

What happened?

After upgrade to Webui 1.8 and try soft inpainting with batch image directory and mask directory, this error happens.

Steps to reproduce the problem

1. Go to webui img2img tab, go to Batch, specifiy Input directory and Inpaint batch mask directory.
2. Then enable Soft inpainting on Genration tab
3. Enable animateDiff and start generate IndexError: index 1 is out of bounds for dimension 0 with size 1 will happen

What should have happened?

If uncheck Soft inpainting, it is work

Commit where the problem happens

webui: 1.8 extension: animateDiff

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

Use webui

Console logs

Traceback (most recent call last):
      File "E:\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "E:\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "E:\stable-diffusion-webui\modules\img2img.py", line 228, in img2img
        processed = process_batch(p, img2img_batch_input_dir, img2img_batch_output_dir, img2img_batch_inpaint_mask_dir, args, to_scale=selected_scale_tab == 1, scale_by=scale_by, use_png_info=img2img_batch_use_png_info, png_info_props=img2img_batch_png_info_props, png_info_dir=img2img_batch_png_info_dir)
      File "E:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 28, in img2img_process_batch_hijack
        return animatediff_i2i_batch(p, *args, **kwargs)
      File "E:\stable-diffusion-webui\extensions\sd-webui-animatediff\scripts\animatediff_i2ibatch.py", line 304, in animatediff_i2i_batch
        proc = process_images(p)
      File "E:\stable-diffusion-webui\modules\processing.py", line 785, in process_images
        res = process_images_inner(p)
      File "E:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 48, in processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "E:\stable-diffusion-webui\modules\processing.py", line 921, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "E:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\hook.py", line 446, in process_sample
        return process.sample_before_CN_hack(*args, **kwargs)
      File "E:\stable-diffusion-webui\modules\processing.py", line 1661, in sample
        samples = self.sampler.sample_img2img(self, self.init_latent, x, conditioning, unconditional_conditioning, image_conditioning=self.image_conditioning)
      File "E:\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 188, in sample_img2img
        samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "E:\stable-diffusion-webui\modules\sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "E:\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 188, in <lambda>
        samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "E:\stable-diffusion-webui\modules\sd_samplers_lcm.py", line 72, in sample_lcm
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "E:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "E:\stable-diffusion-webui\modules\sd_samplers_cfg_denoiser.py", line 198, in forward
        x_in = torch.cat([torch.stack([x[i] for _ in range(n)]) for i, n in enumerate(repeats)] + [x])
      File "E:\stable-diffusion-webui\modules\sd_samplers_cfg_denoiser.py", line 198, in <listcomp>
        x_in = torch.cat([torch.stack([x[i] for _ in range(n)]) for i, n in enumerate(repeats)] + [x])
      File "E:\stable-diffusion-webui\modules\sd_samplers_cfg_denoiser.py", line 198, in <listcomp>
        x_in = torch.cat([torch.stack([x[i] for _ in range(n)]) for i, n in enumerate(repeats)] + [x])
    IndexError: index 1 is out of bounds for dimension 0 with size 1

Additional information

No response

[yuchen1984]

Same problem observed. If turning off "Soft inpainting", the pipeline works fine.

[continue-revolution]

the most possible reason is that there was no soft inpainting and auto has not considered such case for animatediff. Previously I hook everything in cfg denoiser and there was no soft inpainting - even if you turn that on, the code will actually do nothing more. I will try to solve this, but most likely this will result in a pr to a1111 and you will have to wait for webui 1.9 or switch to the dev. stay tuned.

[wildwindfeng]

Thanks for the reply, you are the best and had done the great job.

[continue-revolution]

The reason is that the soft inpaint author made a wrong assumption that there will only be 1 mask, which is not the case for AnimateDiff. I will try to fix and PR to A1111.

[continue-revolution]

If you really want to use soft inpaint with AnimateDiff, please use A1111’s dev branch or wait for v1.9.0.