[Bug]: Batch processing does not work with controlnet (tile) + ultimate sd upscale script, only generates the first image and then stopped

aquCHAOS commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits

What happened?

I tried to do batch up-scaling using ultimate sd upscale script with the controlnet enabled on tile model. The process stops after generating the very first image.

Steps to reproduce the problem

Go to 'Batch' tab in 'img2img'
Enter the 'Input Directory'
Enable 'ControlNet' in 'batch' mode, select Preprocessor as 'tile_resample' and select corresponding tile model.
Select 'Ultimate SD Upscale' script and select the upscale level
Click 'Generate' button

What should have happened?

I expect all the images in the source folder to be processed, instead of the very first image.

I tried to turn off 'ControlNet' or turn off 'Ultimate SD Upscale', in both situation the batch process works as expected.

Commit where the problem happens

version: v1.2.1 • python: 3.10.10 • torch: 1.13.1+cu117 • xformers: 0.0.16rc425 • gradio: 3.29.0 • checkpoint: d7465e52e1

What platforms do you use to access the UI ?

Linux

What browsers do you use to access the UI ?

Mozilla Firefox

Command Line Arguments

python launch.py --no-half-vae --share  --xformers --theme dark --gradio-queue --enable-insecure-extension-access

List of extensions

canvas-zoom https://github.com/richrobber2/canvas-zoom.git be668ffa (Thu May 18 11:28:56 2023) deforum-for-automatic1111-webui https://github.com/deforum-art/deforum-for-automatic1111-webui.git a9aa19b3 (Sat May 20 08:46:25 2023) sd-webui-controlnet https://github.com/Mikubill/sd-webui-controlnet.git bccb1a1d (Sun May 21 13:31:33 2023) ultimate-upscale-for-automatic1111 https://github.com/Coyote-A/ultimate-upscale-for-automatic1111.git 756bb505 (Fri May 5 00:22:21 2023)

Console logs

controlnet batch mode
Will process 5 images, creating 6 new images for each.
Canva size: 1088x1920
Image size: 536x960
Scale factor: 2
Upscaling iteration 1 with scale factor 2
Tile size: 512x512
Tiles amount: 12
Grid: 4x3
Redraw enabled: True
Seams fix mode: NONE
4
Loading model from cache: control_v11f1e_sd15_tile [a371b31b]
Loading preprocessor: tile_resample
preprocessor resolution = 512
100%|███████████████████████████████████████████| 10/10 [00:01<00:00,  6.92it/s]
Loading model from cache: control_v11f1e_sd15_tile [a371b31b]
Loading preprocessor: tile_resample
preprocessor resolution = 512
100%|███████████████████████████████████████████| 10/10 [00:01<00:00,  7.10it/s]
Loading model from cache: control_v11f1e_sd15_tile [a371b31b]
Loading preprocessor: tile_resample
preprocessor resolution = 512
100%|███████████████████████████████████████████| 10/10 [00:01<00:00,  6.42it/s]
Loading model from cache: control_v11f1e_sd15_tile [a371b31b]
Loading preprocessor: tile_resample
preprocessor resolution = 512
100%|███████████████████████████████████████████| 10/10 [00:01<00:00,  6.62it/s]
Loading model from cache: control_v11f1e_sd15_tile [a371b31b]
Loading preprocessor: tile_resample
preprocessor resolution = 512
100%|███████████████████████████████████████████| 10/10 [00:01<00:00,  7.06it/s]
Error completing request
Arguments: ('task(w4vjltlg2tmyjdb)', 5, '', '', [], None, None, None, None, None, None, None, 30, 16, 4, 0, 1, False, False, 1, 6, 7, 1.5, 0.33, -1.0, -1.0, 0, 0, 0, False, 0, 512, 512, 1, 0, 0, 32, 0, '/notebooks/download/test', '', '', [], 11, <controlnet.py.UiControlNetUnit object at 0x7f32d8c9ac50>, '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 3, 2, False, False, False, False, False, False, False, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, None, False, 50, '<p style="margin-bottom:0.75em">Will upscale the image depending on the selected target size type</p>', 512, 0, 8, 32, 64, 0.35, 32, 3, True, 1, False, 8, 0, 2, 2048, 2048, 2) {}
Traceback (most recent call last):
  File "/notebooks/stable-diffusion-webui/modules/call_queue.py", line 57, in f
    res = list(func(*args, **kwargs))
  File "/notebooks/stable-diffusion-webui/modules/call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "/notebooks/stable-diffusion-webui/modules/img2img.py", line 176, in img2img
    process_batch(p, img2img_batch_input_dir, img2img_batch_output_dir, img2img_batch_inpaint_mask_dir, args)
  File "/notebooks/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 30, in img2img_process_batch_hijack
    return getattr(img2img, '__controlnet_original_process_batch')(p, *args, **kwargs)
  File "/notebooks/stable-diffusion-webui/modules/img2img.py", line 67, in process_batch
    proc = modules.scripts.scripts_img2img.run(p, *args)
  File "/notebooks/stable-diffusion-webui/modules/scripts.py", line 408, in run
    processed = script.run(p, *script_args)
  File "/notebooks/stable-diffusion-webui/extensions/ultimate-upscale-for-automatic1111/scripts/ultimate-upscale.py", line 723, in run
    upscaler.process()
  File "/notebooks/stable-diffusion-webui/extensions/ultimate-upscale-for-automatic1111/scripts/ultimate-upscale.py", line 246, in process
    self.image = self.redraw.start(self.p, self.image, self.rows, self.cols)
  File "/notebooks/stable-diffusion-webui/extensions/ultimate-upscale-for-automatic1111/scripts/ultimate-upscale.py", line 378, in start
    return self.chess_process(p, image, rows, cols)
  File "/notebooks/stable-diffusion-webui/extensions/ultimate-upscale-for-automatic1111/scripts/ultimate-upscale.py", line 369, in chess_process
    image = self.chess_process_processing(p, image, rows, cols, True, tiles)
  File "/notebooks/stable-diffusion-webui/extensions/ultimate-upscale-for-automatic1111/scripts/ultimate-upscale.py", line 346, in chess_process_processing
    processed = processing.process_images(p)
  File "/notebooks/stable-diffusion-webui/modules/processing.py", line 526, in process_images
    res = process_images_inner(p)
  File "/notebooks/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 37, in processing_process_images_hijack
    return self.process_images_cn_batch(p, *args, **kwargs)
  File "/notebooks/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 70, in process_images_cn_batch
    self.dispatch_callbacks(self.process_batch_each_callbacks, p)
  File "/notebooks/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 162, in dispatch_callbacks
    callback(*args)
  File "/notebooks/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 1299, in batch_tab_process_each
    unit.image = next(unit.batch_images)
StopIteration

Additional information

No response

bludstorm commented 1 year ago

was there a fix for this or no?

skookie commented 1 year ago

I am having the same problem. With almost the same setup.... why has this been closed? Am I just stupid? This remains broken... anyone with any information would be greatly appreciated.