search

comfyanonymous / ComfyUI

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
https://www.comfy.org/
GNU General Public License v3.0
53.74k stars 5.7k forks source link

Error occurred when using controlnet #970

Closed Njasa2k closed 1 year ago

Njasa2k commented 1 year ago 
!!! Exception during processing !!!
Traceback (most recent call last):
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 144, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 67, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1082, in sample
    return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1052, in common_ksampler
    samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\sample.py", line 88, in sample
    samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 615, in sample
    samples = uni_pc.sample_unipc(self.model_wrap, noise, latent_image, sigmas, sampling_function=sampling_function, max_denoise=max_denoise, extra_args=extra_args, noise_mask=denoise_mask, callback=callback, disable=disable_pbar)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\extra_samplers\uni_pc.py", line 880, in sample_unipc
    x = uni_pc.sample(img, timesteps=timesteps, skip_type="time_uniform", method="multistep", order=order, lower_order_final=True, callback=callback, disable_pbar=disable)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\extra_samplers\uni_pc.py", line 730, in sample
    model_prev_list = [self.model_fn(x, vec_t)]
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\extra_samplers\uni_pc.py", line 421, in model_fn
    return self.data_prediction_fn(x, t)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\extra_samplers\uni_pc.py", line 403, in data_prediction_fn
    noise = self.noise_prediction_fn(x, t)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\extra_samplers\uni_pc.py", line 397, in noise_prediction_fn
    return self.model(x, t)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\extra_samplers\uni_pc.py", line 329, in model_fn
    return noise_pred_fn(x, t_continuous)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\extra_samplers\uni_pc.py", line 297, in noise_pred_fn
    output = model(x, t_input, **model_kwargs)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\external.py", line 98, in predict_eps_discrete_timestep
    return  (input - self(input, sigma, **kwargs)) / utils.append_dims(sigma, input.ndim)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\external.py", line 125, in forward
    eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\k_diffusion\external.py", line 151, in get_eps
    return self.inner_model.apply_model(*args, **kwargs)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 299, in apply_model
    out = sampling_function(self.inner_model.apply_model, x, timestep, uncond, cond, cond_scale, cond_concat, model_options=model_options, seed=seed)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 277, in sampling_function
    cond, uncond = calc_cond_uncond_batch(model_function, cond, uncond, x, timestep, max_total_area, cond_concat, model_options)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 232, in calc_cond_uncond_batch
    c['control'] = control.get_control(input_x, timestep_, c, len(cond_or_uncond))
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\sd.py", line 711, in get_control
    control = self.control_model(x=x_noisy, hint=self.cond_hint, timesteps=t, context=context, y=y)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\cldm\cldm.py", line 311, in forward
    h = module(h, emb, context)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 84, in forward
    x = layer(x, context, transformer_options)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\ldm\modules\attention.py", line 695, in forward
    x = block(x, context=context[i], transformer_options=transformer_options)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\ldm\modules\attention.py", line 527, in forward
    return checkpoint(self._forward, (x, context, transformer_options), self.parameters(), self.checkpoint)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\ldm\modules\diffusionmodules\util.py", line 123, in checkpoint
    return func(*inputs)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\ldm\modules\attention.py", line 627, in _forward
    n = self.attn2(n, context=context_attn2, value=value_attn2)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\ldm\modules\attention.py", line 422, in forward
    k = self.to_k(context)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\*****\ComfyUI\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\ops.py", line 18, in forward
    return torch.nn.functional.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (154x1024 and 768x320)
ltdrdata commented 1 year ago

It seems like using an incompatible model or node with the SDXL model might have caused the issue, but if that's not the case, sharing your workflow would be helpful for diagnosis.

ps. You are using it with controlnet. Currently, there is no controlnet model that supports SDXL. If you have used controlnet in conjunction with SDXL, that is likely the cause of the issue.

Njasa2k commented 1 year ago

oh my bad, I was using the wrong model.

tatbearllc commented 4 weeks ago

well ill try a different model, but the attitude among the tech vs, artist is getting rudder