cloneofsimo / lora

Using Low-rank adaptation to quickly fine-tune diffusion models.
https://arxiv.org/abs/2106.09685
Apache License 2.0
7.03k stars 480 forks source link

Safetensor + lora_add #128

Closed cloneofsimo closed 1 year ago

cloneofsimo commented 1 year ago
  The `patch_pipe` script works for me as well, and the quality is good. However, the lora_add function seems broken. Cannot get it to merge even after converting to ckpt format. For what it's worth, I'm using SD 2.1 as a base. Inference wasn't working on 1.5.

lora_add result with safetensors:

  File "T:\programs\anaconda3\envs\diffusers\lib\site-packages\torch\serialization.py", line 1002, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: could not find MARK

lora_add result with ckpt:

  File "T:\programs\anaconda3\envs\diffusers\lib\site-packages\lora_diffusion\lora.py", line 419, in weight_apply_lora
    up_weight = loras.pop(0).detach().to(weight.device)
KeyError: 0

Originally posted by @Elevory in https://github.com/cloneofsimo/lora/discussions/121#discussioncomment-4636887

cloneofsimo commented 1 year ago

140