vladmandic / automatic

SD.Next: Advanced Implementation of Stable Diffusion and other Diffusion-based generative image models
https://github.com/vladmandic/automatic
GNU Affero General Public License v3.0
5.34k stars 381 forks source link

[Issue]: dev - adetailer + hypertile: introduces an error sometimes, which then sticks for future renders #3165

Open Tillerz opened 1 month ago

Tillerz commented 1 month ago

Issue Description

When using adetailer (and hypertile enabled), at some point SDNext throws an hypertile error (but image rendering continues). From then on, even with adetailer toggled OFF again, the error persists with each render until a full restart has been done.

Version Platform Description

WSL-2, Win11

Relevant log output

14:02:46-468674 DEBUG    Control process unit: i=1 process=None
14:02:46-470713 DEBUG    Pipeline class change: original=StableDiffusionInpaintPipeline target=StableDiffusionPipeline device=cuda:0 fn=control_run
14:02:46-473000 INFO     Applying free-u: b1=1.2 b2=1.4 s1=0.9 s2=0.2
14:02:46-473529 INFO     Applying hypertile: unet=256
14:02:46-476844 INFO     LoRA apply: ['watercolorV1'] patch=0.00 load=0.00
14:02:46-483673 INFO     Base: class=StableDiffusionPipeline
14:02:46-486105 TRACE    Prompt encode: cached
14:02:46-486665 DEBUG    Diffuser pipeline: StableDiffusionPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt_embeds': torch.Size([1, 154,
                         768]), 'negative_prompt_embeds': torch.Size([1, 154, 768]), 'guidance_scale': 3.5, 'num_inference_steps': 30, 'eta': 1.0,
                         'guidance_rescale': 0.7, 'output_type': 'latent', 'width': 512, 'height': 768, 'parser': 'Full parser'}
14:02:46-492153 DEBUG    Sampler: sampler="Euler a" config={'num_train_timesteps': 1000, 'beta_start': 0.00085, 'beta_end': 0.012, 'beta_schedule':
                         'scaled_linear', 'prediction_type': 'epsilon', 'rescale_betas_zero_snr': False, 'timestep_spacing': 'leading'}
14:02:46-503107 DEBUG    HiDiffusion apply: raunet=True attn=True aggressive=True:8 t1=0.2 t2=0.1 time=0.00 type=sd width=512 height=768
Progress 14.97it/s █████████████████████████████████ 100% 30/30 00:02 00:00 Base
14:02:48-529108 DEBUG    Control hires: upscaler="ESRGAN 4x Remacri" upscale=2 size=1024x1536
14:02:48-530109 INFO     Upscale: upscaler="ESRGAN 4x Remacri" resize=0x0 upscale=1024x1536
14:02:49-094528 DEBUG    Upscaler cached: type=ESRGAN model=/home/tillerz/models/ESRGAN/ESRGAN-Remacri-4x.pth
Upscaling ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:01
14:02:50-202806 DEBUG    Pipeline class change: original=StableDiffusionPipeline target=StableDiffusionImg2ImgPipeline device=cuda:0 fn=process_diffusers
14:02:50-203694 INFO     HiRes: class=StableDiffusionImg2ImgPipeline sampler="Same as primary"
14:02:50-212851 TRACE    Prompt encode: cached
14:02:50-213450 DEBUG    Diffuser pipeline: StableDiffusionImg2ImgPipeline task=DiffusersTaskType.IMAGE_2_IMAGE batch=1/1x1 set={'prompt_embeds': torch.Size([1,
                         154, 768]), 'negative_prompt_embeds': torch.Size([1, 154, 768]), 'guidance_scale': 3, 'num_inference_steps': 41, 'eta': 1.0,
                         'output_type': 'latent', 'image': [<PIL.Image.Image image mode=RGB size=1024x1536 at 0x7F70073C1CF0>], 'strength': 0.5, 'parser': 'Full
                         parser'}
Progress ?it/s                                             0% 0/20 00:00 ? Hires14:02:50-933923 ERROR    Hypertile error: width=512 height=768  Error while processing rearrange-reduction pattern "b (nh h nw w) c -> (b nh nw) (h w) c".
                          Input tensor shape: torch.Size([12, 4096, 320]). Additional info: {'h': 26, 'w': 26, 'nh': 3, 'nw': 2}.
                          Shape mismatch, 4096 != 4056
Progress  5.99it/s ████████████████████████████████ 100% 20/20 00:03 00:00 Hires

0: 640x448 (no detections), 622.9ms
Speed: 1.9ms preprocess, 622.9ms inference, 0.4ms postprocess per image at shape (1, 3, 640, 448)
[-] ADetailer: nothing detected on image 1 with 1st settings.
14:02:58-866733 INFO     Processed: images=1 time=12.39 its=2.42 memory={'ram': {'used': 4.59, 'total': 31.35}, 'gpu': {'used': 3.99, 'total': 23.99}, 'retries':
                         0, 'oom': 0}
14:02:59-996518 DEBUG    Save temp: image="/home/tillerz/images_temp/tmpeuihswvs.png" resolution=1024x1536 size=2257125
14:03:00-096707 INFO     Control: pipeline units=0 process=0 time=12.41 init=0.00 proc=0.00 ctrl=12.41 outputs=1

Backend

Diffusers

Branch

Dev

Model

SD 1.5

Acknowledgements

imo9 commented 1 month ago

i have the same persistent hypertile error happening occasionally. for me, it seems to happen when changing the image aspect ratio once. for example, i generate in 512x768 for a while then switch to 768x512 and the hypertile shape mismatch error appears. switching back to 512x768 or other resolutions does not remove the error and the image quality significantly drops.

i could not see a connection to adetailer for me here, but the error seems similar. are you sure it is definitely adetailer causing the issue for you? since you probably also switch image resolutions sometimes too.