Nodes loading ok but workflows not (SUPIR model loader v2 / SUPIR conditioner errors)

[GastonSan79]

First at all, thanks for your work. I'm completely lost here, the nodes load into comfyui ok, but loading the example workflow from the custom node subfolder gives the next error:

Error occurred when executing SUPIR_model_loader_v2:

Failed to load first clip model from SDXL checkpoint

  File "H:\AI\COMFY-PORTABLE\ComfyUI\execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "H:\AI\COMFY-PORTABLE\ComfyUI\execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "H:\AI\COMFY-PORTABLE\ComfyUI\custom_nodes\ComfyUI-0246\utils.py", line 381, in new_func
    res_value = old_func(*final_args, **kwargs)
  File "H:\AI\COMFY-PORTABLE\ComfyUI\execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "H:\AI\COMFY-PORTABLE\ComfyUI\custom_nodes\ComfyUI-SUPIR\nodes_v2.py", line 925, in process
    raise Exception("Failed to load first clip model from SDXL checkpoint")

The loaded model is a SDXL. If I replicate the old example workflow from the repository, the models load ok but at the SUPIR conditioner stage I get the error:

Error occurred when executing SUPIR_conditioner:

Cannot copy out of meta tensor; no data!

  File "H:\AI\COMFY-PORTABLE\ComfyUI\execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "H:\AI\COMFY-PORTABLE\ComfyUI\execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "H:\AI\COMFY-PORTABLE\ComfyUI\custom_nodes\ComfyUI-0246\utils.py", line 381, in new_func
    res_value = old_func(*final_args, **kwargs)
  File "H:\AI\COMFY-PORTABLE\ComfyUI\execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "H:\AI\COMFY-PORTABLE\ComfyUI\custom_nodes\ComfyUI-SUPIR\nodes_v2.py", line 571, in condition
    SUPIR_model.conditioner.to(device)
  File "H:\AI\COMFY-PORTABLE\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1160, in to
    return self._apply(convert)
  File "H:\AI\COMFY-PORTABLE\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    module._apply(fn)
  File "H:\AI\COMFY-PORTABLE\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    module._apply(fn)
  File "H:\AI\COMFY-PORTABLE\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    module._apply(fn)
  File "H:\AI\COMFY-PORTABLE\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 833, in _apply
    param_applied = fn(param)
  File "H:\AI\COMFY-PORTABLE\python_embeded\lib\site-packages\torch\nn\modules\module.py", line 1158, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)

All the nodes are correctly connected. Python version: 3.10.11 / ComfyUI Revision: 2021 [36f7face] | Released on '2024-02-26

Thanks in advance.

[Amit30swgoh]

Error occurred when executing SUPIR_sample:

No operator found for memory_efficient_attention_forward with inputs: query : shape=(40, 126, 1, 64) (torch.float16) key : shape=(40, 126, 1, 64) (torch.float16) value : shape=(40, 126, 1, 64) (torch.float16) attn_bias : p : 0.0 decoderF is not supported because: xFormers wasn't build with CUDA support attn_bias type is operator wasn't built - see python -m xformers.info for more info flshattF@0.0.0 is not supported because: xFormers wasn't build with CUDA support operator wasn't built - see python -m xformers.info for more info cutlassF is not supported because: xFormers wasn't build with CUDA support operator wasn't built - see python -m xformers.info for more info smallkF is not supported because: max(query.shape[-1] != value.shape[-1]) > 32 xFormers wasn't build with CUDA support dtype=torch.float16 (supported: {torch.float32}) operator wasn't built - see python -m xformers.info for more info unsupported embed per head: 64

File "/content/drive/MyDrive/ComfyUI/execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "/content/drive/MyDrive/ComfyUI/execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-0246/utils.py", line 381, in new_func res_value = old_func(final_args, kwargs) File "/content/drive/MyDrive/ComfyUI/execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(slice_dict(input_data_all, i))) File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes_v2.py", line 481, in sample _samples = self.sampler(denoiser, noised_z, cond=positive[i], uc=negative[i], x_center=sample.unsqueeze(0), control_scale=control_scale_end, File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/sampling.py", line 441, in call x = self.sampler_step( File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/sampling.py", line 418, in sampler_step denoised = self.denoise(x, denoiser, sigma_hat, cond, uc, control_scale=control_scale) File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/sampling.py", line 400, in denoise denoised = denoiser(self.guider.prepare_inputs(x, sigma, cond, uc), control_scale) File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes_v2.py", line 455, in denoiser = lambda input, sigma, c, control_scale: SUPIR_model.denoiser(SUPIR_model.model, input, sigma, c, control_scale) File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/denoiser.py", line 73, in call return network(input c_in, c_noise, cond, control_scale) c_out + input c_skip File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/diffusionmodules/wrappers.py", line 96, in forward out = self.diffusion_model( File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/SUPIR/modules/SUPIR_v0.py", line 654, in forward h = self.project_modules[adapter_idx](control[control_idx], h, control_scale=control_scale) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/SUPIR/modules/SUPIR_v0.py", line 147, in forward x = self.attn(x, context) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, **kwargs) File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/attention.py", line 365, in forward out = xformers.ops.memory_efficient_attention( File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/init.py", line 247, in memory_efficient_attention return _memory_efficient_attention( File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/init.py", line 365, in _memory_efficient_attention return _memory_efficient_attention_forward( File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/init.py", line 381, in _memory_efficient_attention_forward op = _dispatch_fw(inp, False) File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/dispatch.py", line 125, in _dispatch_fw return _run_priority_list( File "/usr/local/lib/python3.10/dist-packages/xformers/ops/fmha/dispatch.py", line 65, in _run_priority_list raise NotImplementedError(msg)

[tapistefan]

Error occurred when executing SUPIR_model_loader_v2:

Try going into Manager and "Update All". It fixed the error for me.

[kijai]

I don't really know what could cause this, I can't replicate it, current version with the example workflow runs fine for me.

[Sostay]

Exactly the same situation, when I use the model of fp16.

[SRagy]

I also have issues with these nodes, although the error messages are not the same. On the first time I queue something, I get the following error for the loader:

Error occurred when executing SUPIR_model_loader_v2:

Failed to load SDXL model

File "/notebooks/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/notebooks/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/notebooks/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/notebooks/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes_v2.py", line 893, in process
raise Exception("Failed to load SDXL model")

On the second time, this error disappears, which is a bit perplexing, but I become subsequently stuck at the conditioner with the following error:

Error occurred when executing SUPIR_conditioner:

'NoneType' object is not callable

File "/notebooks/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/notebooks/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/notebooks/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/notebooks/ComfyUI/custom_nodes/ComfyUI-SUPIR/nodes_v2.py", line 617, in condition
_c, _uc = SUPIR_model.conditioner.get_unconditional_conditioning(cond, uncond)
File "/notebooks/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/encoders/modules.py", line 187, in get_unconditional_conditioning
c = self(batch_c)
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/notebooks/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/encoders/modules.py", line 208, in forward
emb_out = embedder(batch[embedder.input_key])
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/notebooks/ComfyUI/custom_nodes/ComfyUI-SUPIR/sgm/modules/encoders/modules.py", line 490, in forward
batch_encoding = self.tokenizer(

This happens when running the sample workflow, so I don't think it's a problem with how I've set the nodes up. My python environment is potentially a bit janky though, because I use paperspace which has some inflexibility.

[moguldigital]

@SRagy Did you find some solution to this? I'm having this same issue.

[Sostay]

@SRagy Did you find some solution to this? I'm having this same issue.

It seems that there are some environment-dependent problems. I adjusted the network environment and sometimes it can work normally.

[moguldigital]

@SRagy I'm using runpod with 3090 gpu, can you tell what things you changed because I can't figure it out. I also don't know coding much but if you can give a direction of what to do then I can chatgpt my way through it.

[SRagy]

In the end I used the legacy node from V1, which worked fine for me.

[firofame]

seems duplicate of https://github.com/kijai/ComfyUI-SUPIR/issues/72