[Issue]: Upscaling & Inpainting producing strange error

[HarrisTerry]

Issue Description

Yesterday I had this error while upscaling ESRGANx4 Exception: Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 192, 192] to have 3 channels, but got 4 channels instead

Today while inpainting, I got all this:

023-05-25 09:05:24,807 | sd | WARNING | postprocessing_upscale | Could not find upscaler: <empty string>

2023-05-25 09:05:25,740 | sd | ERROR | call_queue | Exception: Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 192, 192] to have 3 channels, but got 4 channels instead

2023-05-25 09:05:25,740 | sd | ERROR | call_queue | Arguments: args=('task(uwl63zjostikc4n)', 2, 'photo of 25-year old Columbian woman, 1girl, face, half body, body, high detailed skin, skin pores, nude,  8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3, inspired by (StacyKeibler2:1.071) and (ravenlyn:1.021) and (PiperSaint:1.011) ', '(semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime, stylized:1.4), text, close up, cropped, out of frame, worst quality, low quality, jpeg artifacts, ugly, duplicate, morbid, mutilated, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, fused fingers, too many fingers, long neck', [], None, None, {'image': <PIL.Image.Image image mode=RGBA size=4000x4000 at 0x22D866D9D20>, 'mask': <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=4000x4000 at 0x22D866D8700>}, None, None, None, None, 30, 15, 4, 0, 1, False, False, 1, 1, 7, 1.5, 1, 0.48, -1.0, -1.0, 0, 0, 0, False, 0, 512, 512, 1, 0, 1, 32, 0, '', '', '', [], 0, 1, 0.2, 0, 0, 4, 512, 512, True, 'None', 'None', 0, 'None', False, False, 240, 10, 10, False, 'MultiDiffusion', False, True, 1024, 1024, 96, 96, 48, 1, 'None', 2, False, 10, 1, 1, 64, False, False, False, False, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 1536, 96, True, True, True, False, False, 7, 100, 'Constant', 0, 'Constant', 0, 4, False, 'x264', 'blend', 10, 0, 0, False, True, True, True, 'intermediate', 'animation', <controlnet.py.UiControlNetUnit object at 0x0000022D866D8370>, <controlnet.py.UiControlNetUnit object at 0x0000022E150B6050>, <controlnet.py.UiControlNetUnit object at 0x0000022D866DFDF0>, <controlnet.py.UiControlNetUnit object at 0x0000022D866DD000>, False, '', 0.5, True, False, '', 'Lerp', False, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, None, None, False, 50, False, 4.0, '', 10.0, 'Linear', 3, False, 30.0, True, False, False, 0, 0.0, 'Lanczos', 1, True, 0, 0, 0.001, 75, 0.0, False, True, '<p style="margin-bottom:0.75em">Will upscale the image depending on the selected target size type</p>', 512, 0, 8, 32, 64, 0.35, 32, 0, True, 0, False, 8, 0, 0, 2048, 2048, 2) kwargs={}

2023-05-25 09:05:25,757 | sd | ERROR | errors | gradio call: RuntimeError

Version Platform Description

Win11, 12700k 5ghz, 3070ti 8GB, 64GB RAM, Edge

2023-05-25 09:02:27,842 | sd | INFO | installer | Version: fc82ea2d Thu May 25 08:51:46 2023 -0400

[HarrisTerry]

Got this today trying inpainting in a batch of 10

10:12:21-683336 WARNING  Could not find upscaler: <empty string>
10:12:22-747766 ERROR    Exception: Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 192, 192] to have
                         3 channels, but got 4 channels instead
10:12:22-748770 ERROR    Arguments: args=('task(f44khkgh7bbjr2u)', 2, 'photo of 20-year old red haired woman, 1girl,
                         face, half body, body, (fit:0.9), (large breasts), body, high detailed skin, skin pores, 8k
                         uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3, inspired by ',
                         '(semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime, stylized:1.4), text, close
                         up, cropped, out of frame, worst quality, low quality, jpeg artifacts, ugly, duplicate, morbid,
                         mutilated, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation,
                         deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face,
                         disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra
                         legs, fused fingers, too many fingers, long neck', [], None, None, {'image': <PIL.Image.Image
                         image mode=RGBA size=4000x4000 at 0x25A92050B80>, 'mask': <PIL.PngImagePlugin.PngImageFile
                         image mode=RGBA size=4000x4000 at 0x25A920516F0>}, None, None, None, None, 26, 15, 4, 0, 1,
                         False, False, 10, 1, 7, 1.5, 1, 0.41, -1.0, -1.0, 0, 0, 0, False, 0, 512, 512, 1, 0, 1, 32, 0,
                         '', '', '', [], 0, 1, 0.2, 0, 0, 4, 512, 512, True, 'None', 'None', 0, 'None', False, False,
                         240, 10, 10, False, 'MultiDiffusion', False, True, 1024, 1024, 96, 96, 48, 1, 'None', 2, False,
                         10, 1, 1, 64, False, False, False, False, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2,
                         -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2,
                         '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0,
                         False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '',
                         'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False,
                         0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 1536, 96, True, True, True, False,
                         False, 7, 100, 'Constant', 0, 'Constant', 0, 4, False, 'x264', 'blend', 10, 0, 0, False, True,
                         True, True, 'intermediate', 'animation', <controlnet.py.UiControlNetUnit object at
                         0x0000025A920E3EE0>, <controlnet.py.UiControlNetUnit object at 0x0000025A9BB2A710>,
                         <controlnet.py.UiControlNetUnit object at 0x0000025A915D3820>, <controlnet.py.UiControlNetUnit
                         object at 0x0000025A92159C90>, False, '', 0.5, True, False, '', 'Lerp', False,
                         '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '',
                         True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '<p
                         style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a,
                         Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0,
                         ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '<p
                         style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and
                         height sliders to set tile size</p>', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False,
                         False, False, 0, False, None, None, False, None, None, False, None, None, False, None, None,
                         False, 50, False, 4.0, '', 10.0, 'Linear', 3, False, 30.0, True, False, False, 0, 0.0,
                         'Lanczos', 1, True, 0, 0, 0.001, 75, 0.0, False, True, '<p style="margin-bottom:0.75em">Will
                         upscale the image depending on the selected target size type</p>', 512, 0, 8, 32, 64, 0.35, 32,
                         0, True, 0, False, 8, 0, 0, 2048, 2048, 2) kwargs={}
10:12:22-755769 ERROR    gradio call: RuntimeError
╭───────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────╮
│ D:\AI\Vlad1111\automatic\modules\call_queue.py:34 in f                                                               │
│                                                                                                                      │
│    33 │   │   │   try:                                                                                               │
│ ❱  34 │   │   │   │   res = func(*args, **kwargs)                                                                    │
│    35 │   │   │   │   progress.record_results(id_task, res)                                                          │
│                                                                                                                      │
│ D:\AI\Vlad1111\automatic\modules\img2img.py:161 in img2img                                                           │
│                                                                                                                      │
│   160 │   │   if processed is None:                                                                                  │
│ ❱ 161 │   │   │   processed = process_images(p)                                                                      │
│   162 │   p.close()                                                                                                  │
│                                                                                                                      │
│                                               ... 17 frames hidden ...                                               │
│                                                                                                                      │
│ D:\AI\Vlad1111\automatic\venv\lib\site-packages\torch\nn\modules\conv.py:463 in forward                              │
│                                                                                                                      │
│    462 │   def forward(self, input: Tensor) -> Tensor:                                                               │
│ ❱  463 │   │   return self._conv_forward(input, self.weight, self.bias)                                              │
│    464                                                                                                               │
│                                                                                                                      │
│ D:\AI\Vlad1111\automatic\venv\lib\site-packages\torch\nn\modules\conv.py:459 in _conv_forward                        │
│                                                                                                                      │
│    458 │   │   │   │   │   │   │   _pair(0), self.dilation, self.groups)                                             │
│ ❱  459 │   │   return F.conv2d(input, weight, bias, self.stride,                                                     │
│    460 │   │   │   │   │   │   self.padding, self.dilation, self.groups)                                             │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 192, 192] to have 3 channels, but got 4
channels instead

[vladmandic]

please format the log entries when you're copying and pasting, makes it impossible to read otherwise. there is a well established standard how to quote paste blocks.

this error is still under investigation and does happen sometimes, but so far no full root cause has been established. what appears so far is that error occurs just before out-of-memory, for example if you're trying to upscale to too high of a resolution or performing batch operations that exceed available memory.

[HarrisTerry]

Interesting. I had just restarted the program, and the error isn't happening in Auto1111 with the exact same extension set up. I have 8GB VRAM, and it is just basic inpainting on 512x512 masked area. I will test with smaller batches.

I will check out how to format logs, I have not posted them in the past much :P

[HarrisTerry]

This time I have no log as while doing a small batch generation of just text to image, my whole computer locked up as if a process was running and nothing else could, the mouse was stuttering, and I had to reboot as I could not even get to task manager. It was very modest, 512x512 pics in batches of 5, I have done batches of 40 upscaled in the past.

[Razunter]

Maybe unrelated, but check if "Apply color correction to img2img results to match original colors" option causes this.

[vladmandic]

@HarrisTerry lockup/extreme slowdown is likely #1285

i'm still any closer to understanding original error

RuntimeError: Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 192, 192] to have 3 channels, but got 4 channels instead

[HarrisTerry]

@vladmandic yes sir, rolled back to 531 and no issues, closed! :) thanks!

[vladmandic]

i can no longer reproduce original issue, but i've added extra logging in that area just in case. if issue reoccurs, please run webui --debug and post the log here and i'll reopen.