search

vladmandic / automatic

SD.Next: Advanced Implementation of Stable Diffusion and other Diffusion-based generative image models
https://github.com/vladmandic/automatic
GNU Affero General Public License v3.0
5.56k stars 409 forks source link

[Diffusers]: Inpaint result not match the size of the source image #1930

Closed ShmuelRonen closed 1 year ago

ShmuelRonen commented 1 year ago

Issue Description

Hello... I made an text2image in size of 896 x 1152. I transfer it to inpaint. but the results is always in distorted square. that not match the original size.

20230804075849-on the Twin Towers inpaint

Diffusers pipeline used

Stable Diffusion XL and Stable Diffusion XL inpaint

Version Platform Description

Starting SD.Next 22:12:46-160604 INFO Python 3.10.7 on Windows 22:12:46-253613 INFO Version: 90f6cc8e Wed Jul 26 20:54:51 2023 -0400 Windows 10 pro

Relevant log output

C:\V\automatic>webui.bat --backend diffusers --medvram --experimental
Using VENV: C:\V\automatic\venv
22:12:46-156606 INFO     Starting SD.Next
22:12:46-160604 INFO     Python 3.10.7 on Windows
22:12:46-253613 INFO     Version: 90f6cc8e Wed Jul 26 20:54:51 2023 -0400
22:12:46-675605 INFO     Latest published version: 434a1f967f41fcaabfa6af796106951be00851fd 2023-08-03T18:06:15Z
22:12:46-679606 INFO     nVidia CUDA toolkit detected
22:12:46-925603 INFO     Disabled extensions: ['a1111-sd-webui-lycoris', 'sd-dynamic-thresholding',
                         'multidiffusion-upscaler-for-automatic1111', 'sd-webui-controlnet']
22:12:46-928603 INFO     Enabled extensions-builtin: ['clip-interrogator-ext', 'LDSR', 'Lora', 'ScuNET',
                         'sd-extension-system-info', 'sd-webui-agent-scheduler',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR']
22:12:46-934606 INFO     Enabled extensions: ['animator_extension', 'deforum-for-automatic1111-webui',
                         'infinite-zoom-automatic1111-webui', 'Mask2Background', 'sd-face-editor',
                         'sd-webui-aspect-ratio-helper']
22:12:46-940602 INFO     Verifying requirements
22:12:46-958602 INFO     Verifying packages
22:12:46-961603 INFO     Verifying repositories
22:12:51-396871 INFO     Verifying submodules
22:13:16-244580 ERROR    Error running extension installer: C:\V\automatic\extensions\animator_extension\install.py
22:13:33-280580 INFO     Extensions enabled: ['clip-interrogator-ext', 'LDSR', 'Lora', 'ScuNET',
                         'sd-extension-system-info', 'sd-webui-agent-scheduler',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR',
                         'animator_extension', 'deforum-for-automatic1111-webui',
                         'infinite-zoom-automatic1111-webui', 'Mask2Background', 'sd-face-editor',
                         'sd-webui-aspect-ratio-helper']
22:13:33-286577 INFO     Verifying packages
22:13:33-288577 WARNING  Setup complete with errors: 1
22:13:33-289577 WARNING  See log file for more details: C:\V\automatic\sdnext.log
22:13:33-299580 INFO     Extension preload: 0.0s C:\V\automatic\extensions-builtin
22:13:33-300578 INFO     Extension preload: 0.0s C:\V\automatic\extensions
22:13:33-318578 INFO     Server arguments: ['--backend', 'diffusers', '--medvram', '--experimental']
No module 'xformers'. Proceeding without it.
22:13:39-925577 INFO     Pipeline: Backend.DIFFUSERS
22:13:41-620577 INFO     Libraries loaded
22:13:41-622580 INFO     Using data path: C:\V\automatic
22:13:41-626577 INFO     Available VAEs: C:\V\automatic\models\VAE 3
22:13:41-675577 INFO     Available models: C:\V\automatic\models\Stable-diffusion 5
22:13:45-144577 INFO     Libraries loaded
22:13:46-642783 INFO     Loading UI theme: name=black-orange style=Auto
Running on local URL:  http://127.0.0.1:7860
22:13:50-080807 INFO     Local URL: http://127.0.0.1:7860/
22:13:50-082806 INFO     Initializing middleware
22:13:50-181806 INFO     [AgentScheduler] Task queue is empty
22:13:50-183806 INFO     [AgentScheduler] Registering APIs
22:13:50-289804 INFO     Setting Torch parameters: dtype=torch.float16 vae=torch.float16 unet=torch.float16
22:13:50-293805 INFO     Loading diffuser model: stabilityai/stable-diffusion-xl-base-1.0
Loading pipeline components...: 100%|██████████████████████████████████████████████████| 7/7 [00:02<00:00,  3.14it/s]
22:14:02-153428 INFO     Embeddings: loaded=0 skipped=0
22:14:02-156430 INFO     Model loaded in 11.9s (load=11.9s) native=1024
22:14:02-476430 INFO     Model load finished: {'ram': {'used': 1.11, 'total': 31.24}, 'gpu': {'used': 8.22, 'total':
                         24.0}, 'retries': 0, 'oom': 0}
22:14:02-483437 INFO     Setting Torch parameters: dtype=torch.float16 vae=torch.float16 unet=torch.float16
22:14:02-485430 INFO     Loading diffuser refiner: stabilityai/stable-diffusion-xl-refiner-1.0
Loading pipeline components...: 100%|██████████████████████████████████████████████████| 5/5 [00:01<00:00,  3.69it/s]
------------

                         kwargs={}
10:27:46-461595 ERROR    gradio call: ValueError
┌──────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────┐
│ C:\V\automatic\modules\call_queue.py:34 in f                                                                      │
│                                                                                                                   │
│    33 │   │   │   try:                                                                                            │
│ >  34 │   │   │   │   res = func(*args, **kwargs)                                                                 │
│    35 │   │   │   │   progress.record_results(id_task, res)                                                       │
│                                                                                                                   │
│ C:\V\automatic\modules\img2img.py:186 in img2img                                                                  │
│                                                                                                                   │
│   185 │   │   if processed is None:                                                                               │
│ > 186 │   │   │   processed = processing.process_images(p)                                                        │
│   187 │   p.close()                                                                                               │
│                                                                                                                   │
│                                              ... 2 frames hidden ...                                              │
│                                                                                                                   │
│ C:\V\automatic\modules\processing.py:1055 in init                                                                 │
│                                                                                                                   │
│   1054 │   │   │   │   image_masked = Image.new('RGBa', (image.width, image.height))                              │
│ > 1055 │   │   │   │   image_masked.paste(image.convert("RGBA").convert("RGBa"), mask=ImageOps.                   │
│   1056 │   │   │   │   self.mask = image_mask # assign early for diffusers                                        │
│                                                                                                                   │
│ C:\V\automatic\venv\lib\site-packages\PIL\Image.py:1732 in paste                                                  │
│                                                                                                                   │
│   1731 │   │   │   mask.load()                                                                                    │
│ > 1732 │   │   │   self.im.paste(im, box, mask.im)                                                                │
│   1733 │   │   else:                                                                                              │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
ValueError: images do not match
100%|████████████████████████████████████████████████████████████████████████████████| 15/15 [00:06<00:00,  2.26it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 15/15 [00:06<00:00,  2.26it/s]
10:41:08-386542 INFO     Setting Torch parameters: dtype=torch.float16 vae=torch.float16 unet=torch.float16
10:41:08-389542 INFO     Loading diffuser model: stabilityai/stable-diffusion-xl-base-1.0
Loading pipeline components...: 100%|██████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.35it/s]
10:41:21-560940 INFO     Embeddings: loaded=0 skipped=0
10:41:21-562943 INFO     Model loaded in 13.2s (load=13.2s) native=1024
10:41:21-962779 INFO     Model load finished: {'ram': {'used': 0.64, 'total': 31.24}, 'gpu': {'used': 8.41, 'total':
                         24.0}, 'retries': 0, 'oom': 0}
10:41:45-672684 INFO     Pipeline class changed from StableDiffusionXLPipeline to StableDiffusionXLInpaintPipeline
100%|████████████████████████████████████████████████████████████████████████████████| 15/15 [00:06<00:00,  2.27it/s]

Acknowledgements

ShadoWxShinigamI commented 1 year ago

I am having the same issue. Inpaint always gives me a square image no matter what the resize settings are when using "whole image" and using "only masked" also resizes the image to a square before inpainting which causes the inpainted results to not be good. OnlyMasked WholeImage

vladmandic commented 1 year ago

fixed, will post in 1h.