coder168
commented
5 months ago I did little more tracing, what I found as follows:
- I remove "--disable-nan-check" from startup script, and repeat the same testing steps, the problem as same as before, but this time "NansException" (instead of a warning) raised in logging:
venv "D:\app2\stable-diffusion-webui\venv\Scripts\Python.exe" Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] Version: v1.9.3 Commit hash: 1c0a0c4c26f78c32095ebc7f8af82f5c04fca8c0 Launching Web UI with arguments: --medvram --opt-split-attention --xformers --ckpt-dir D:\app2\sd-models\Stable-diffusion --lora-dir D:\app2\sd-models\Lora --embeddings-dir D:\app2\sd-models\embeddings --controlnet-dir D:\app2\sd-models\controlnet ControlNet preprocessor location: D:\app2\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads 2024-05-05 23:20:25,163 - ControlNet - INFO - ControlNet v1.1.447 2024-05-05 23:20:25,447 - ControlNet - INFO - ControlNet v1.1.447 Loading weights [879db523c3] from D:\app2\sd-models\Stable-diffusion\dreamshaper_8.safetensors 2024-05-05 23:20:26,177 - ControlNet - INFO - ControlNet UI callback registered. Creating model from config: D:\app2\stable-diffusion-webui\configs\v1-inference.yaml Running on local URL: http://127.0.0.1:7860
To create a public link, set share=True in launch().
Startup time: 24.6s (prepare environment: 4.5s, import torch: 10.6s, import gradio: 1.2s, setup paths: 1.1s, initialize shared: 0.4s, other imports: 0.9s, list SD models: 0.4s, load scripts: 3.9s, create ui: 1.2s, gradio launch: 0.4s).
Applying attention optimization: xformers... done.
Model loaded in 4.3s (load weights from disk: 0.9s, create model: 1.8s, apply weights to model: 1.0s, calculate empty prompt: 0.5s).
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00, 3.75it/s]
tiled upscale: 100%|███████████████████████████████████████████████████████████████████| 15/15 [00:03<00:00, 4.82it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 10/10 [00:08<00:00, 1.23it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 30/30 [00:23<00:00, 1.25it/s]
2024-05-05 23:23:14,109 - ControlNet - INFO - unit_separate = False, style_align = False 30/30 [00:23<00:00, 1.25it/s]
2024-05-05 23:23:14,378 - ControlNet - INFO - Using preprocessor: reference_only
2024-05-05 23:23:14,379 - ControlNet - INFO - preprocessor resolution = 512
2024-05-05 23:23:14,780 - ControlNet - INFO - ControlNet Hooked - Time = 0.6819639205932617
0%| | 0/20 [00:00<?, ?it/s]2024-05-05 23:23:16,929 - ControlNet - INFO - ControlNet used torch.float16 VAE to encode torch.Size([1, 4, 96, 64])./s]
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:12<00:00, 1.57it/s]
tiled upscale: 100%|███████████████████████████████████████████████████████████████████| 15/15 [00:03<00:00, 4.88it/s]
0%| | 0/10 [00:00<?, ?it/s]2024-05-05 23:23:35,053 - ControlNet - INFO - ControlNet used torch.float16 VAE to encode torch.Size([1, 4, 192, 128]).
100%|██████████████████████████████████████████████████████████████████████████████████| 10/10 [00:20<00:00, 2.08s/it]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 30/30 [00:43<00:00, 1.46s/it]
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:04<00:00, 4.01it/s]
tiled upscale: 100%|███████████████████████████████████████████████████████████████████| 15/15 [00:03<00:00, 4.87it/s]
0%| | 0/10 [00:01<?, ?it/s]
Error completing request
Arguments: ('task(kbsgfg8stzpcs7p)', <gradio.routes.Request object at 0x000001FAF2186710>, 'masterpiece, best quality, photography, 8K, HDR,\n1woman, beautiful, white background,', 'BadDream, UnrealisticDream, By bad artist -neg, badhandv4, ng_deepnegative_v1_75t,\nworst quality, low quality, normal quality,', [], 1, 1, 7, 768, 512, True, 0.4, 2, '4x-UltraSharp', 10, 0, 0, 'Use same checkpoint', 'DPM++ 2M', 'Use same scheduler', '', '', [], 0, 20, 'Euler a', 'Automatic', False, '', 0.8, 123456, False, -1, 0, 0, 0, False, 0.0, 4, 0.0, 512, 512, True, 'None', 'None', 0, False, 1, 0, False, 1, UiControlNetUnit(enabled=False, module='reference_only', model='None', weight=1, image={'image': array([[[218, 221, 222],
[219, 220, 224],
[218, 221, 224],
...,
[208, 212, 215],
[207, 211, 216],
[208, 209, 214]],
[[218, 220, 223], [219, 221, 224], [218, 220, 224], ..., [209, 211, 215], [208, 210, 216], *** [207, 210, 216]],
[[217, 220, 223], [218, 221, 224], [217, 220, 223], ..., [209, 212, 217], [209, 211, 215], *** [209, 210, 214]],
*** ...,
[[198, 200, 204], [196, 199, 203], [196, 200, 203], ..., [197, 203, 208], [196, 202, 207], *** [197, 202, 208]],
[[197, 200, 203], [196, 200, 202], [196, 200, 203], ..., [195, 202, 206], [196, 201, 206], *** [196, 201, 207]],
[[197, 199, 203], [195, 199, 202], [197, 199, 204], ..., [195, 201, 206], [195, 202, 206], [196, 200, 204]]], dtype=uint8), 'mask': array([[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], *** [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], *** [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], *** [0, 0, 0]],
*** ...,
[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], *** [0, 0, 0]],
[[0, 0, 0], [0, 0, 0], [0, 0, 0], ..., [0, 0, 0], [0, 0, 0], *** [0, 0, 0]],
[[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
...,
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]], dtype=uint8)}, resize_mode='Crop and Resize', low_vram=True, processor_res=512, threshold_a=0.5, threshold_b=0.5, guidance_start=0, guidance_end=1, pixel_perfect=True, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None, effective_region_mask=None, pulid_mode='Fidelity', ipadapter_input=None), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None, effective_region_mask=None, pulid_mode='Fidelity', ipadapter_input=None), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None, effective_region_mask=None, pulid_mode='Fidelity', ipadapter_input=None), False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50) {}
Traceback (most recent call last):
File "D:\app2\stable-diffusion-webui\modules\call_queue.py", line 57, in f
res = list(func(args, kwargs))
File "D:\app2\stable-diffusion-webui\modules\call_queue.py", line 36, in f
res = func(*args, kwargs)
File "D:\app2\stable-diffusion-webui\modules\txt2img.py", line 109, in txt2img
processed = processing.process_images(p)
File "D:\app2\stable-diffusion-webui\modules\processing.py", line 845, in process_images
res = process_images_inner(p)
File "D:\app2\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 59, in processing_process_images_hijack
return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, kwargs)
File "D:\app2\stable-diffusion-webui\modules\processing.py", line 981, in process_images_inner
samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
File "D:\app2\stable-diffusion-webui\modules\processing.py", line 1344, in sample
return self.sample_hr_pass(samples, decoded_samples, seeds, subseeds, subseed_strength, prompts)
File "D:\app2\stable-diffusion-webui\modules\processing.py", line 1429, in sample_hr_pass
samples = self.sampler.sample_img2img(self, samples, noise, self.hr_c, self.hr_uc, steps=self.hr_second_pass_steps or self.steps, image_conditioning=image_conditioning)
File "D:\app2\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 172, in sample_img2img
samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, extra_params_kwargs))
File "D:\app2\stable-diffusion-webui\modules\sd_samplers_common.py", line 272, in launch_sampling
return func()
File "D:\app2\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 172, in
- by sending a new request(in a new browser tab) with different settings & prompt using hires step, sometime it works again. When it work again, it works for any prompts including the previous one which did get errors. It seems somehow the new request cleanup certain internal mess-up left by the task with controlnet.
huchenlei
Is there an existing issue for this?
What happened?
This is a very wired bug. It seems controlnet can interfere hires step somehow even after controlnet is disabled. It is tricky, it seems controlnet cause certain impacts which has not been completed restored after disabled. Please investigate and let us know what you found, I am quite curious.
Steps to reproduce the problem
What should have happened?
3rd run should not broken. Since it is the exactly same input and settings like 1st run. If 1st run works fine, 3rd run should get exactly same results. The only difference is 3rd run after 2nd run which controlnet enable.
Commit where the problem happens
webui: version: v1.9.3 controlnet: v1.1.446
What browsers do you use to access the UI ?
Google Chrome
Command Line Arguments
List of enabled extensions
Console logs
Additional information