search

shiimizu / ComfyUI-TiledDiffusion

Tiled Diffusion, MultiDiffusion, Mixture of Diffusers, and optimized VAE
294 stars 21 forks source link

Using the example Simple and 4x Upscale workflows results in an error when using SDXL #14

Open Jellybit opened 8 months ago

Jellybit commented 8 months ago

I feel like I must be doing something wrong since no one else is encountering this. I tried to look for any special settings I must add/change to use SDXL, but nothing was mentioned in the readme. This occurs when using either the Simple Upscale or 4x Upscale workflow examples in the readme.

Here's an example, where you can turn on/off Simple or 4x: Simple and 4X Test UpscaleSimpleAnd4x.json

  1. Hook up an SDXL model/latent, set to 1024x1024.
  2. Change the Upscale Image node to 2048x2048, following the example upscale amount in the original workflow (included this in case it's necessary for it to work properly, even though the bug happens without this step).
  3. Click on Queue to generate.

Instead of completing the upscale, it results in this error:

Error occurred when executing KSamplerAdvanced:

mat1 and mat2 shapes cannot be multiplied (385x2048 and 768x320)

File "Q:\SD\ComfyUI\Portable\ComfyUI\execution.py", line 155, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\execution.py", line 85, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\execution.py", line 78, in map_node_over_list results.append(getattr(obj, func)(slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\nodes.py", line 1389, in sample return common_ksampler(model, noise_seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise, disable_noise=disable_noise, start_step=start_at_step, last_step=end_at_step, force_full_denoise=force_full_denoise) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\nodes.py", line 1325, in common_ksampler samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 22, in informative_sample raise e File "Q:\SD\ComfyUI\Portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 9, in informative_sample return original_sample(*args, *kwargs) # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 241, in motion_sample return orig_comfy_sample(model, noise, args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\sample.py", line 100, in sample samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\custom_nodes\ComfyUI_smZNodes__init.py", line 130, in KSampler_sample return _KSampler_sample(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\samplers.py", line 716, in sample return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\custom_nodes\ComfyUI_smZNodes\init.py", line 149, in sample return _sample(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\samplers.py", line 622, in sample samples = sampler.sample(model_wrap, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\samplers.py", line 561, in sample samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, self.extra_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\k_diffusion\sampling.py", line 503, in sample_dpmpp_2s_ancestral denoised = model(x, sigmas[i] * s_in, *extra_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\samplers.py", line 285, in forward out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, model_options=model_options, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\samplers.py", line 275, in forward return self.apply_model(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\custom_nodes\ComfyUI_smZNodes\smZNodes.py", line 1030, in apply_model out = super().apply_model(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\samplers.py", line 272, in apply_model out = sampling_function(self.inner_model, x, timestep, uncond, cond, cond_scale, model_options=model_options, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\samplers.py", line 252, in sampling_function cond_pred, uncond_pred = calc_cond_uncondbatch(model, cond, uncond, x, timestep, model_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\custom_nodes\ComfyUI-TiledDiffusion.patches.py", line 95, in calc_cond_uncond_batch output = model_options['model_function_wrapper'](model.apply_model, {"input": inputx, "timestep": timestep, "c": c, "cond_or_uncond": cond_or_uncond}).chunk(batch_chunks) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\custom_nodes\ComfyUI-TiledDiffusion\tiled_diffusion.py", line 528, in call__ c_tile['control'] = control.get_control(x_tile, t_tile, c_tile, len(cond_or_uncond)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\controlnet.py", line 176, in get_control control = self.control_model(x=x_noisy.to(dtype), hint=self.cond_hint, timesteps=timestep.float(), context=context.to(dtype), y=y) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\cldm\cldm.py", line 305, in forward h = module(h, emb, context) ^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 62, in forward return forward_timestep_embed(self, *args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 46, in forward_timestep_embed x = layer(x, context, transformer_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\ldm\modules\attention.py", line 616, in forward x = block(x, context=context[i], transformer_options=transformer_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\ldm\modules\attention.py", line 443, in forward return checkpoint(self._forward, (x, context, transformer_options), self.parameters(), self.checkpoint) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\ldm\modules\diffusionmodules\util.py", line 189, in checkpoint return func(inputs) ^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\ldm\modules\attention.py", line 543, in _forward n = self.attn2(n, context=context_attn2, value=value_attn2) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\ldm\modules\attention.py", line 387, in forward k = self.to_k(context) ^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\ComfyUI\comfy\ops.py", line 27, in forward return super().forward(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "Q:\SD\ComfyUI\Portable\python_embeded\Lib\site-packages\torch\nn\modules\linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

shiimizu commented 8 months ago

You were using an SD1.5 ControlNet model with an SDXL checkpoint.

Either switch to an SD1.5 checkpoint or use an SDXL ControlNet model here (rank 256) or here (rank 128) 1 Preprocessors can be found here.

There's currently no "tile" for SDXL, but there are alternatives in ControlNet-LLLite such as blur or tile_anime.

Jellybit commented 8 months ago

Oh thank you! and sorry for not figuring this out myself. I appreciate it. Hopefully this will be found by others who make the same mistake I did.