comfyanonymous / ComfyUI

The most powerful and modular stable diffusion GUI, api and backend with a graph/nodes interface.
GNU General Public License v3.0
41.15k stars 4.38k forks source link

can't load lcm checkpoint, lcm lora works well #1933

Open frankyifei opened 7 months ago

frankyifei commented 7 months ago

when trying to load lcm sdxl checkpoint, throw this error

Error occurred when executing CheckpointLoaderSimple:

'model.diffusion_model.input_blocks.0.0.weight'

File "F:\Comfy\ComfyUI\execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "F:\Comfy\ComfyUI\execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "F:\Comfy\ComfyUI\execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "F:\Comfy\ComfyUI\nodes.py", line 476, in load_checkpoint
out = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, embedding_directory=folder_paths.get_folder_paths("embeddings"))
File "F:\Comfy\ComfyUI\comfy\sd.py", line 431, in load_checkpoint_guess_config
model_config = model_detection.model_config_from_unet(sd, "model.diffusion_model.", unet_dtype)
File "F:\Comfy\ComfyUI\comfy\model_detection.py", line 141, in model_config_from_unet
unet_config = detect_unet_config(state_dict, unet_key_prefix, dtype)
File "F:\Comfy\ComfyUI\comfy\model_detection.py", line 49, in detect_unet_config
model_channels = state_dict['{}input_blocks.0.0.weight'.format(key_prefix)].shape[0]
comfyanonymous commented 7 months ago

It's in diffusers format so you have to use the UNETLoader node to load it.

Here's a workflow: ComfyUI_07672_

frankyifei commented 7 months ago

Thank you, it works. I also noticed there is a big difference in speed when I changed CFG to 1.0, 2.56/s. if using higher or lower than 1, speed is only around 1.40 which is what I normally get with SDXL. Same thing happen to lcm-lora. Why is that?

comfyanonymous commented 7 months ago

CFG 1.0 means there's negative prompt so there's 2x less processing to do to generate the image.

zhanghongyong123456 commented 7 months ago

It's in diffusers format so you have to use the UNETLoader node to load it.

Here's a workflow: ComfyUI_07672_

when i use this workflow, i have this error, image I use UNERLoader model : download model in
image

image I'm not sure what's wrong

zhanghongyong123456 commented 7 months ago

It's in diffusers format so you have to use the UNETLoader node to load it. Here's a workflow: ComfyUI_07672_

when i use this workflow, i have this error, image I use UNERLoader model : download model in image

image I'm not sure what's wrong

Mybe use this worlflow ,it's OK,Use Load Lora replace https://github.com/comfyanonymous/ComfyUI/issues/1934#issuecomment-1804991327

comfyanonymous commented 7 months ago

Use this workflow for the LCM loras: https://gist.github.com/comfyanonymous/b948fd68aacc65c2ed37d743d0964f27

zhanghongyong123456 commented 7 months ago

Use this workflow for the LCM loras: https://gist.github.com/comfyanonymous/b948fd68aacc65c2ed37d743d0964f27

thanks, It is OK ,

zhanghongyong123456 commented 7 months ago

Use this workflow for the LCM loras: https://gist.github.com/comfyanonymous/b948fd68aacc65c2ed37d743d0964f27

How do you use these two models( 2 Full Param-tuned LCM (SD-XL, SSD-1B),): https://huggingface.co/latent-consistency/lcm-sdxl

image

longkenj commented 7 months ago

image i get this error: When loading the graph, the following node types were not found: ModelSamplingDiscrete Nodes that have failed to load will show as red on the graph.

where can i download the node?

ltdrdata commented 7 months ago

image i get this error: When loading the graph, the following node types were not found: ModelSamplingDiscrete Nodes that have failed to load will show as red on the graph.

where can i download the node?

Update ComfyUI

zhangp365 commented 7 months ago

image When I use this in video 2 video, there are some error logs. How can I fix that? image workflow (2).json

xiaocaijiayou commented 7 months ago

It's in diffusers format so you have to use the UNETLoader node to load it.

Here's a workflow: ComfyUI_07672_

Where to get the unet file? I only have the whole safetensors file with all components.

giusparsifal commented 1 month ago

Hell, sorry for my question, what does it means "workflow is in the image"? I can't understand it, please someone will explain me? Thank you...

Dont-look-at-me commented 2 weeks ago

when i tried to use SD3, i encounted the same problom. i dont know what's wrong.

the error code: !!! Exception during processing!!! 'model.diffusion_model.input_blocks.0.0.weight' Traceback (most recent call last): File "D:\SD3\UI\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\SD3\UI\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\SD3\UI\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\SD3\UI\ComfyUI_windows_portable\ComfyUI\nodes.py", line 516, in load_checkpoint out = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, embedding_directory=folder_paths.get_folder_paths("embeddings")) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\SD3\UI\ComfyUI_windows_portable\ComfyUI\comfy\sd.py", line 476, in load_checkpoint_guess_config model_config = model_detection.model_config_from_unet(sd, "model.diffusion_model.") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\SD3\UI\ComfyUI_windows_portable\ComfyUI\comfy\model_detection.py", line 232, in model_config_from_unet unet_config = detect_unet_config(state_dict, unet_key_prefix) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\SD3\UI\ComfyUI_windows_portable\ComfyUI\comfy\model_detection.py", line 113, in detect_unet_config model_channels = state_dict['{}input_blocks.0.0.weight'.format(key_prefix)].shape[0]


KeyError: 'model.diffusion_model.input_blocks.0.0.weight'