Uplay111
commented
10 months ago how ABOUT controlnet's tile resample model in sdxl?
Open lllyasviel opened 10 months ago
Uplay111
commented
10 months ago how ABOUT controlnet's tile resample model in sdxl?
Kr01iKs
commented
10 months ago hi collaborators I just tested Tiled Diffusion/VAE with SD 1.5. It worked without problem. Just for clarification.
Didn't it work before?
WhiteZeroX
commented
10 months ago Hmmm I update today to v1.1.407 and if I enable Tiled Diffusion with an upscale ControNet errors out. I was using whatever ControlNet version released yesterday with no issues. EDIT: If I change the Latent tile width and height to 128, ControlNet then doesnt error out. But I've been using 96 for ages, not sure why it broken now?
--- MixtureOfDiffusers Sampling: 0%| | 0/17 [00:19<?, ?it/s] [Tiled Diffusion] upscaling image with 4x-UltraSharp... [Tiled Diffusion] ControlNet found, support is enabled. MixtureOfDiffusers Sampling: 0%| | 0/17 [00:00<?, ?it/s]Mixture of Diffusers hooked into 'Euler' sampler, Tile size: 96x96, Tile batches: 1, Batch size: 4. (ext: ContrlNet) [Tiled VAE]: input_size: torch.Size([1, 3, 1024, 1024]), tile_size: 512, padding: 32 [Tiled VAE]: split to 2x2 = 4 tiles. Optimal tile size 480x480, original tile size 512x512 [Tiled VAE]: Executing Encoder Task Queue: 100%|████████████████████████████████████| 364/364 [00:00<00:00, 443.30it/s] [Tiled VAE]: Done in 1.094s, max VRAM alloc 737.833 MB█████████████████▋ | 240/364 [00:00<00:00, 467.03it/s] 2023-09-06 10:54:49,545 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b] 2023-09-06 10:54:49,547 - ControlNet - INFO - Loading preprocessor: tile_resample 2023-09-06 10:54:49,547 - ControlNet - INFO - preprocessor resolution = -1 2023-09-06 10:54:49,563 - ControlNet - INFO - ControlNet Hooked - Time = 0.01901721954345703 0%| | 0/17 [00:00<?, ?it/s] *** Error completing request | 0/17 [00:00<?, ?it/s] *** Arguments: ('task(bbn54k96p0c8f13)', 0, 'beautiful woman', '', [], <PIL.Image.Image image mode=RGBA size=512x512 at 0x21D9BF805B0>, None, None, None, None, None, None, 25, 'Euler', 4, 0, 1, 1, 1, 5, 1.5, 0.65, 1, 512, 512, 2, 0, 0, 32, 0, '', '', '', [], False, [], '', <gradio.routes.Request object at 0x0000021EA6C8DBD0>, 0, False, '', 0.8, -1, False, -1, 0, 0, 0, True, 'Mixture of Diffusers', False, True, 1024, 1024, 96, 96, 32, 4, '4x-UltraSharp', 2, False, 10, 1, 1, 64, True, False, False, False, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, True, 512, 64, True, False, False, False, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000021EA6C8C730>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000021EA6C8F7C0>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000021EA6C8F400>, '*CFG Scale` should be 2 or lower.', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '
Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8
', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', 'Will upscale the image by the selected scale factor; use width and height sliders to set tile size
', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50) {} Traceback (most recent call last): File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\call_queue.py", line 57, in f res = list(func(*args, kwargs)) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\call_queue.py", line 36, in f res = func(*args, *kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\img2img.py", line 208, in img2img processed = process_images(p) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\processing.py", line 732, in process_images res = process_images_inner(p) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack return getattr(processing, '__controlnet_original_process_images_inner')(p, args, kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\processing.py", line 867, in process_images_inner samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\hook.py", line 451, in process_sample return process.sample_before_CN_hack(*args, kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\processing.py", line 1528, in sample samples = self.sampler.sample_img2img(self, self.init_latent, x, conditioning, unconditional_conditioning, image_conditioning=self.image_conditioning) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 188, in sample_img2img samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, extra_params_kwargs)) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\sd_samplers_common.py", line 261, in launch_sampling return func() File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 188, in---`
thesimpleway1
commented
10 months ago Hmmm I update today to v1.1.407 and if I enable Tiled Diffusion with an upscale ControNet errors out. I was using whatever ControlNet version released yesterday with no issues. EDIT: If I change the Latent tile width and height to 128, ControlNet then doesnt error out. But I've been using 96 for ages, not sure why it broken now?
--- MixtureOfDiffusers Sampling: 0%| | 0/17 [00:19<?, ?it/s] [Tiled Diffusion] upscaling image with 4x-UltraSharp... [Tiled Diffusion] ControlNet found, support is enabled. MixtureOfDiffusers Sampling: 0%| | 0/17 [00:00<?, ?it/s]Mixture of Diffusers hooked into 'Euler' sampler, Tile size: 96x96, Tile batches: 1, Batch size: 4. (ext: ContrlNet) [Tiled VAE]: input_size: torch.Size([1, 3, 1024, 1024]), tile_size: 512, padding: 32 [Tiled VAE]: split to 2x2 = 4 tiles. Optimal tile size 480x480, original tile size 512x512 [Tiled VAE]: Executing Encoder Task Queue: 100%|████████████████████████████████████| 364/364 [00:00<00:00, 443.30it/s] [Tiled VAE]: Done in 1.094s, max VRAM alloc 737.833 MB█████████████████▋ | 240/364 [00:00<00:00, 467.03it/s] 2023-09-06 10:54:49,545 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b] 2023-09-06 10:54:49,547 - ControlNet - INFO - Loading preprocessor: tile_resample 2023-09-06 10:54:49,547 - ControlNet - INFO - preprocessor resolution = -1 2023-09-06 10:54:49,563 - ControlNet - INFO - ControlNet Hooked - Time = 0.01901721954345703 0%| | 0/17 [00:00<?, ?it/s] *** Error completing request | 0/17 [00:00<?, ?it/s] *** Arguments: ('task(bbn54k96p0c8f13)', 0, 'beautiful woman', '', [], <PIL.Image.Image image mode=RGBA size=512x512 at 0x21D9BF805B0>, None, None, None, None, None, None, 25, 'Euler', 4, 0, 1, 1, 1, 5, 1.5, 0.65, 1, 512, 512, 2, 0, 0, 32, 0, '', '', '', [], False, [], '', <gradio.routes.Request object at 0x0000021EA6C8DBD0>, 0, False, '', 0.8, -1, False, -1, 0, 0, 0, True, 'Mixture of Diffusers', False, True, 1024, 1024, 96, 96, 32, 4, '4x-UltraSharp', 2, False, 10, 1, 1, 64, True, False, False, False, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, True, 512, 64, True, False, False, False, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000021EA6C8C730>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000021EA6C8F7C0>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000021EA6C8F400>, '*CFG Scale` should be 2 or lower.', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', 'Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8 ', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '
Will upscale the image by the selected scale factor; use width and height sliders to set tile size ', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50) {} Traceback (most recent call last): File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\call_queue.py", line 57, in f res = list(func(*args, kwargs)) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\call_queue.py", line 36, in f res = func(*args, *kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\img2img.py", line 208, in img2img processed = process_images(p) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\processing.py", line 732, in process_images res = process_images_inner(p) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack return getattr(processing, '__controlnet_original_process_images_inner')(p, args, kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\processing.py", line 867, in process_images_inner samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\hook.py", line 451, in process_sample return process.sample_before_CN_hack(*args, kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\processing.py", line 1528, in sample samples = self.sampler.sample_img2img(self, self.init_latent, x, conditioning, unconditional_conditioning, image_conditioning=self.image_conditioning) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 188, in sample_img2img samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, extra_params_kwargs)) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\sd_samplers_common.py", line 261, in launch_sampling return func() File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 188, in samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, extra_params_kwargs)) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\sampling.py", line 128, in sample_euler denoised = model(x, sigma_hat s_in, extra_args) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\sd_samplers_cfg_denoiser.py", line 169, in forward x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict(cond_in, image_cond_in)) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\external.py", line 112, in forward eps = self.get_eps(input c_in, self.sigma_to_t(sigma), kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\external.py", line 138, in get_eps return self.inner_model.apply_model(*args, kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\extensions\multidiffusion-upscaler-for-automatic1111\tile_utils\utils.py", line 243, in wrapper return fn(args, kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\extensions\multidiffusion-upscaler-for-automatic1111\tile_methods\mixtureofdiffusers.py", line 131, in apply_model_hijack x_tile_out = shared.sd_model.apply_model_original_md(x_tile, t_tile, c_tile) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\sd_hijack_utils.py", line 17, in setattr(resolved_obj, func_path[-1], lambda *args, kwargs: self(*args, kwargs)) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\modules\sd_hijack_utils.py", line 28, in call* return self.__orig_func(args, kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 858, in apply_model x_recon = self.model(x_noisy, t, cond) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1538, in _call_impl result = forward_call(*args, *kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 1335, in forward out = self.diffusion_model(x, t, context=cc) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\hook.py", line 853, in forward_webui raise e File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\hook.py", line 850, in forward_webui return forward(*args, kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\hook.py", line 591, in forward control = param.control_model(x=x_in, hint=hint, timesteps=timesteps, context=context, y=y) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\cldm.py", line 31, in forward return self.control_model(args, kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "C:\Games\Misc\stablediffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\cldm.py", line 311, in forward h += guided_hint RuntimeError: The size of tensor a (96) must match the size of tensor b (128) at non-singleton dimension 3
---`
I have the same issue with tile.
I was always using 128 as latent width and high but it still gives me an runtime error.
lllyasviel
commented
10 months ago fixed in 1.1.408
WhiteZeroX
commented
10 months ago fixed in 1.1.408
Confirmed! Thank you!
Amorano
commented
9 months ago This error has returned for me in version 1.1.411.
The temp fix is the same: Set latent Tile Width and Height to 128
simplete
commented
8 months ago 我在 1.1.411 版本中返回了此错误。
临时修复是相同的：将潜在图块宽度和高度设置为 128
me too
hi collaborators I just tested Tiled Diffusion/VAE with SD 1.5. It worked without problem. Just for clarification.