Cannot merge lora(s) into a checkpoint in v21.8.6

[michaeltran33]

Cannot merge lora(s) into a checkpoint in v21.8.6 Console. 15:27:05-153681 INFO Version: v21.8.6

15:27:05-160681 INFO nVidia toolkit detected 15:27:05-905680 INFO Torch 2.0.1+cu118 15:27:05-928705 INFO Torch backend: nVidia CUDA 11.8 cuDNN 8700 15:27:05-932705 INFO Torch detected GPU: NVIDIA GeForce RTX 3060 VRAM 12288 Arch (8, 6) Cores 28 15:27:05-933706 INFO Verifying modules instalation status from requirements_windows_torch2.txt... 15:27:05-935706 INFO Verifying modules instalation status from requirements.txt... 15:27:07-517731 INFO headless: False 15:27:07-520732 INFO Load CSS... Running on local URL: http://127.0.0.1:7861/

To create a public link, set share=True in launch(). 15:27:30-683045 INFO Merge model... 15:27:30-684045 INFO ./venv/Scripts/python.exe "networks\sdxl_merge_lora.py" --sd_model "D:/stable-diffusion-webui/models/Stable-diffusion/AModel/lofi2.safetensors" --save_precision fp16 --precision float --save_to "D:/stable-diffusion-webui/models/Stable-diffusion/AModel/lofi3.safetensors" --models "D:/stable-diffusion-webui/models/Lora/Specials/NijiV3Fantasy.safetensors" "D:/stable-diffusion-webui/models/Lora/Specials/ocIllustration_v10.safetensors" --ratios 0.4 0.4 A matching Triton is not available, some optimizations will not be enabled. Error caught was: No module named 'triton' loading SD model: D:/stable-diffusion-webui/models/Stable-diffusion/AModel/lofi2.safetensors building U-Net loading U-Net from checkpoint ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ D:\kohya_ss\networks\sdxl_merge_lora.py:257 in │ │ │ │ 254 │ parser = setup_parser() │ │ 255 │ │ │ 256 │ args = parser.parse_args() │ │ ❱ 257 │ merge(args) │ │ 258 │ │ │ │ D:\kohya_ss\networks\sdxl_merge_lora.py:205 in merge │ │ │ │ 202 │ │ │ unet, │ │ 203 │ │ │ logit_scale, │ │ 204 │ │ │ ckpt_info, │ │ ❱ 205 │ │ ) = sdxl_model_util.load_models_from_sdxl_checkpoint(sdxl_model_util.MODEL_VERSI │ │ 206 │ │ │ │ 207 │ │ merge_to_sd_model(text_model1, text_model2, unet, args.models, args.ratios, merg │ │ 208 │ │ │ │ D:\kohya_ss\library\sdxl_model_util.py:196 in load_models_from_sdxl_checkpoint │ │ │ │ 193 │ for k in list(state_dict.keys()): │ │ 194 │ │ if k.startswith("model.diffusion_model."): │ │ 195 │ │ │ unet_sd[k.replace("model.diffusion_model.", "")] = state_dict.pop(k) │ │ ❱ 196 │ info = _load_state_dict_on_device(unet, unet_sd, device=map_location) │ │ 197 │ print("U-Net: ", info) │ │ 198 │ │ │ 199 │ # Text Encoders │ │ │ │ D:\kohya_ss\library\sdxl_model_util.py:158 in _load_state_dict_on_device │ │ │ │ 155 │ if unexpected_keys: │ │ 156 │ │ error_msgs.insert(0, "Unexpected key(s) in state_dict: {}. ".format(", ".join('" │ │ 157 │ │ │ ❱ 158 │ raise RuntimeError("Error(s) in loading state_dict for {}:\n\t{}".format(model.__cla │ │ 159 │ │ 160 │ │ 161 def load_models_from_sdxl_checkpoint(model_version, ckpt_path, map_location, dtype=None) │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ RuntimeError: Error(s) in loading state_dict for SdxlUNet2DConditionModel: Unexpected key(s) in state_dict: "output_blocks.11.1.transformer_blocks.0.attn1.to_q.weight", "output_blocks.11.1.transformer_blocks.0.attn2.to_v.weight", "output_blocks.9.0.out_layers.3.weight", "output_blocks.8.2.conv.weight", "input_blocks.2.1.transformer_blocks.0.attn2.to_k.weight", "output_blocks.11.1.proj_out.bias", "output_blocks.10.1.norm.weight", "output_blocks.11.1.transformer_blocks.0.norm3.weight", "output_blocks.9.0.out_layers.3.bias", "output_blocks.9.0.in_layers.0.weight", "output_blocks.11.1.transformer_blocks.0.norm1.bias",

[kohya-ss]

Please verify the model is SDXL based. If the model is SD1/2 based, please use merge_lora.py instead.

[michaeltran33]

I think i know the problem. the gui removed the merge_lora.py and replaced it with the sdxl_merge_lora.py and uses it instead, even the model is sd15 based. I wonder how I can change the gui to generate the right model output. Any suggestion wd be grateful.

[kohya-ss]

I think you can run the command directory in the command prompt or PowerShell, with modifying sdxl_merge_lora.py to merge_lora.py.

[michaeltran33]

Thank you. I found the issue in the gui n changed file library/merge_lora_gui.py line 71 from "if not sd_model: to if sd_model:" and the gui gens cmd correctly. Thanks for the guide and help.

[kohya-ss]

That's nice! Perhaps you could report this issue to the repo of GUI :)