Lora keys not loaded

[Mark-papi]

ENVIRONMENT Windows 10 GPU 1660 Super 32 gb ram

So i tried a lora model that i made, and i try to get results from prompts but i get an warning lora keys not loaded and the image is not the desired

[comfyanonymous]

Can you paste what these lora keys are?

[ra100]

Seems I have similar issue, I've used this to train my LoRA: https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/README_sdxl.md

and using text2img from that page yields completely different results, as if loading in ComfyUI

This works fine:

from diffusers import DiffusionPipeline
import torch

model_path = "takuoko/sd-pokemon-model-lora-sdxl"
pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16)
pipe.to("cuda")
pipe.load_lora_weights(model_path)

prompt = "A pokemon with green eyes and red legs."
image = pipe(prompt, num_inference_steps=30, guidance_scale=7.5).images[0]
image.save("pokemon.png")

Can I mimic it in ComfyUI? I'm don't have access to machine where I've run it right now, I'll post the message, missing keys later, I can share the model I trained with prompts too.

[ra100]

I'm getting these:

left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.out_proj.lora_linear_layer.up.weight

and if it helps, here is produced pytorch_lora_weights.bin file https://drive.google.com/file/d/16-Mr7wZM2S4SJP06XX2UtFHgYC21192n/view?usp=sharing

[comfyanonymous]

Should be fixed with: https://github.com/comfyanonymous/ComfyUI/commit/c20583286fca8a55324ee7d60cebd3b511b16c91

[ra100]

Thank you, I've just tried it, no errors thrown and result looks as expected.

[Alx-AI]

Getting the same error: lora key not loaded down_blocks.1.attentions.0.transformer_blocks.0.attn1.processor.to_k_lora.down.weight ...

with the following two Lora's: The first is a 'wrong' Lora

The second is trained with this process and uploaded here

[comfyanonymous]

Make sure you update/update_comfyui.bat on the standalone or git pull and they probably will work.

[Alx-AI]

You were right, sorry. thank you!

[ra100]

@Mark-papi is it fixed for you too? I guess this issue could be closed.

[ra100]

Hi @comfyanonymous, seems the issue is back, maybe because of the finetuning reimplementation using PEFT in diffusers (https://github.com/huggingface/diffusers/pull/6225)

newly trained LoRAs don't work in comfy.

partial log:

lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.k_proj.lora_A.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.k_proj.lora_B.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.out_proj.lora_A.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.out_proj.lora_B.weight

Here is the full list of keys it has issues with: keys.txt

He're the created LoRA: pytorch_lora_weights.safetensors.zip (trigger should be "fsfrnsl man")

[Philomorph]

Hi @comfyanonymous, seems the issue is back, maybe because of the finetuning reimplementation using PEFT in diffusers (huggingface/diffusers#6225)

newly trained LoRAs don't work in comfy.

partial log:

lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.k_proj.lora_A.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.k_proj.lora_B.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.out_proj.lora_A.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.out_proj.lora_B.weight

I'm having this same issue, but I just installed ComfyUI and started using it yesterday. Was using A1111 with no problems and my LORAs were all downloaded a month or so ago - don't know when they were trained.

But then I downloaded a new LORA I hadn't tried before and the error went away and it seems to work.

[DavidPetit]

I am also having these issues. I used this notebook to train my Lora: https://github.com/huggingface/notebooks/blob/main/diffusers/SDXL_DreamBooth_LoRA_.ipynb

The Lora works fine when testing it using CLI or inside this notebook. However, in ComfyUI, I got a lot of those errors: lora key not loaded unet.down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_k.lora.down.weight

I hope it can be fixed someday because I really love using ComfyUI. :)

[Pierian-Data]

Am also getting the lora key not loaded set of messages on a new install of ComfyUI

[rainstormads]

I'm getting this error as well. Any fix to this?

[radames]

Just for reference got this issued resolved by converting my latest LoRA weights from diffusers to ComfyUI using this script https://github.com/huggingface/diffusers/blob/main/scripts/convert_diffusers_sdxl_lora_to_webui.py, maybe these dict changes could be integrated to ComfyUI?

https://github.com/huggingface/diffusers/blob/357855f8fca8d27beb0a3fde333f46db22f29391/src/diffusers/utils/state_dict_utils.py#L25-L113

[burghman24]

@radames Pleas pardon me for knowing nothing (I'm very new to this world) but when I try to run that script, I keep getting an error on ln 25 because the module "diffusers" (used as "from diffusers.utils import [two specific functions]). This does not seem to be the standard "diffusers" library that I can pip install because those oddly named functions are not included in the library. Could you provide any guidance for a total noob?

[radames]

hi @burghman24 what's the diffusers version? you can try 0.25.1 or install and use direct from main pip install git+https://github.com/huggingface/diffusers.git

[burghman24]

@radames I appreciate the reply and it did fix my initial problem. Just for a reference, ReaPhoLoRA.safetensors is the specific lora that I was trying to use via Comfy UI that is causing the errors the OP created this thread to discuss.

But this lora does not seem to convert via the script that works for you. I keep returning ValueErrors when I'm trying to run it. Not sure if the format of my specific lora is not compatible with this script -- or if I am just doing it wrong (which is ALWAYS a possibility since I am green as the grass at all of this). Because the SDXLrender_v2.0.safetensors lora produced the same errors in ComfyUI, I also tried to convert it and got the same ValueErrors. This one references the convert_all_state_dict_to_peft function. I think I may just be a bit out of my depth on this one...

[OhBobb]

@radames and @bughman24

I apologize for also being newby on this and for long post but this is what I tried so far: -when I tried command: "python convert_diffusers_sdxl_lora_to_webui.py c:\users\admin\downloads\Dark_Futuristic_Circuit_Boards.safetensors" (this is the lora im trying to work/convert), I got this result:

"Traceback (most recent call last):
   File "C:\users\admin\downloads\convert_diffusers_sdxl_lora_to_webui.py", line 25, in <module>
       from diffusers.utils import convert_all_state_dict_to_perf, convert_state_dict_to_kohya
ImportError: cannot import name 'convert_all_state_dict_to_peft' from 'diffusers.utils' (c:\program files\python311\lib\site-packages\diffusers\utils\__ini__.py)"

-I tried running the following commands to install different versions of diffusers: "pip install diffusers==0.18.2" , "pip install diffusers==0.20.1" and also "pip install diffusers==0.25.1" and continue to get same error as above

-I tried this command(from @radames): "pip install git+https://github.com/huggingface/diffusers.git" , and the error went away but instead got this error:

"Traceback (most recent call last):
   File "C:\users\admin\downloads\convert_diffusers_sdxl_lora_to_webui.py", line 55, in <module>
      convert_and_save(args.input_lora, args.output_lora)
   File "c:\users\admin\downloads\convert_diffusers_sdxl_lora_to_webui.py", line 34, in convert_and_save
      peft_state_dict = convert_all_state_dict_to_peft(diffusers_state_dict)

   File "c:\program files\python311\lib\site-packages\diffusers\utils\state_dict_utils.py", line 262, in convert_all_stat
e_dict_to_peft
      raise ValueError("Your LoRA was not converted to PEFT")
ValueError: Your LoRA was not converted to PEFT"

Is it because the lora is not a compatible lora or am I not doing something correctly with the command?

[burghman24]

@OhBobb @radames -- That is precisely the same error I am getting now too. It appears we reached the same point where things break down. If anyone does have a solution, I am most anxious to hear it!

[Pierian-Data]

I also got the same error after wrangling around with the imports, and I even use a LoRa script mentioned in the notebook file, so who knows what the underlying issue is

[Fractaz]

Hi, Very sorry...... know this is closed but I'm having same issue with my 1.5 Lora's - have followed the problem this far and am looking at the python code above and thinking, great, maybe I need to do that. Plus my Lora's aren't SDXL ones. BUT I've not the slightest idea how to use it, and just in case I'm barking up wrong tree I will paste a bit of my error code. Any help would be much appreciated, I'm fine with nodes, just lost with code, google got me this far.

got prompt model_type EPS adm 0 Using xformers attention in VAE Working with z of shape (1, 4, 32, 32) = 4096 dimensions. Using xformers attention in VAE missing {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'} left over keys: dict_keys(['embedding_manager.embedder.transformer.text_model.embeddings.position_embedding.weight', 'embedding_manager.embedder.transformer.text_model.embeddings.position_ids', 'embedding_manager.embedder.transformer.text_model.embeddings.token_embedding.weight', 'embedding_manager.embedder.transformer.text_model.encoder.layers.0.layer_norm1.bias', 'embedding_manager.embedder.transformer.text_model.encoder.layers.0.layer_norm1.weight', 'embedding_manager.embedder.transformer.text_model.encoder.layers.0.layer_norm2.bias', 'embedding_manager.embedder.transformer.text_model.e

and goes on and on.

[jloveric]

I found the converter convert_diffusers_sdxl_lora_to_webui.py also works for sd1.5 loras. I used hugging face to create the loras.

[ikpark09]

I faced the same issue. In my case 'convert_diffusers_sdxl_lora_to_webui.py' didn't work. after tried many thing finally solved the problems. Don't use the 'LoadLoraModelOnly' node, use 'Load Lora' node and connect the clip node. then 'Lora keys not loaded' and 'missing Clip blah blah' will disappear and Lora works well.

[Suka4GitHub]

I'm getting a similar error as follows;

lora key not loaded: lora_unet_down_blocks_0_attentions_0_proj_in.alpha lora key not loaded: lora_unet_down_blocks_0_attentions_0_proj_in.lora_down.weight lora key not loaded: lora_unet_down_blocks_0_attentions_0_proj_in.lora_up.weight lora key not loaded: lora_unet_down_blocks_0_attentions_0_proj_out.alpha lora key not loaded: lora_unet_down_blocks_0_attentions_0_proj_out.lora_down.weight lora key not loaded: lora_unet_down_blocks_0_attentions_0_proj_out.lora_up.weight lora key not loaded: lora_unet_down_blocks_0_attentions_0_proj_out.lora_down.weight lora key not loaded: lora_unet_down_blocks_0_attentions_0_proj_out.lora_up.weight lora key not loaded: lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_k.alpha lora key not loaded: lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_k.lora_down.weight lora key not loaded: lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_k.lora_up.weight lora key not loaded: lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_out_0.alpha lora key not loaded: lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_down.weight lora key not loaded: lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_up.weight lora key not loaded: lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_q.alpha

Any suggestions would be much appreciated thank you in advance!

PS. Use SDXL loras for SDXL etc. x

[GittyCat1]

Should be fixed with: c205832

how do I append a commit? I couldn't figure out how to use git commit. I opened up comfy/sd.py, tried using notepadd++ to compare, and it seems they are not the same file at all. I assume the green lines in github commit page were to be added to my file, and white should have been the same, but several white lines where not even in the file. I got the error using wildcardx-xl with forRealXL_v3 (both civitai). Pinoko comfyai.

[Omarsmsm]

I fixed mine because I realized I get checkout'ed with a different branch from XLABS and didnt switch back to be able to update from master branch. So once I pointed .git back to master everything worked.