cloneofsimo / lora

Using Low-rank adaptation to quickly fine-tune diffusion models.
https://arxiv.org/abs/2106.09685
Apache License 2.0
6.94k stars 479 forks source link

the trained safetensors does not work in Automatic1111 WebUI #212

Open harrywang opened 1 year ago

harrywang commented 1 year ago

I used the following script to train a style and the final model is in final_lora.safetensors

lora_pti \
  --pretrained_model_name_or_path=$MODEL_NAME  \
  --instance_data_dir=$INSTANCE_DIR \
  ...
Screenshot 2023-03-03 at 7 48 50 PM

I use the code https://github.com/cloneofsimo/lora/blob/master/scripts/run_inference.ipynb to generate result and it works - the style is applied.

I want to use https://github.com/AUTOMATIC1111/stable-diffusion-webui to generate result and put the safetensors file in models/Lora/ but run into the following error in log:

activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x11e1393f0>]: ValueError
Traceback (most recent call last):
  File "/Users/harrywang/sandbox/stable-diffusion-webui/modules/extra_networks.py", line 75, in activate
    extra_network.activate(p, extra_network_args)
  File "/Users/harrywang/sandbox/stable-diffusion-webui/extensions-builtin/Lora/extra_networks_lora.py", line 23, in activate
    lora.load_loras(names, multipliers)
  File "/Users/harrywang/sandbox/stable-diffusion-webui/extensions-builtin/Lora/lora.py", line 151, in load_loras
    lora = load_lora(name, lora_on_disk.filename)
  File "/Users/harrywang/sandbox/stable-diffusion-webui/extensions-builtin/Lora/lora.py", line 89, in load_lora
    key, lora_key = fullkey.split(".", 1)
ValueError: not enough values to unpack (expected 2, got 1)

I think the key is not correct similar to the bin lora weights trained using Diffusers as discussed https://github.com/huggingface/diffusers/issues/2326 - which needs to be converted for WebUI.

I wonder how we can do that for lora_pit trained lora weights.

Thanks a lot!!

harrywang commented 1 year ago

https://github.com/cloneofsimo/lora/issues/149 is related.

tengshaofeng commented 1 year ago

@harrywang @anotherjesse @zeke @levi . I have add some code to support the Automatic111 WebUI. What you should do is just runing 'train_lora_dreambooth.py', then you will find that there is another lora_weight_webui.safetensors in the output dir. That is the right weight for WebUI. reference my project in https://github.com/tengshaofeng/lora_tbq