Lora appears to be created during training but once complete it doesn't produce expected results[Bug]:

klubkelli commented 5 months ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

I trained a Lora using images of a person. During training, it seems like it's working on when it's done with the training. I go to use the image, and it doesn't produce images of the girl.

This is an early training image at about 2700 steps. Not great, but clearly, it is the expected result.

sample_2450-0

When the training is complete, and I got to use the lora it created, it doesn't work. The file appeared to have worked during training. But not when it saves.

Training finished. Total lifetime steps: 3500

I run the prompt "photo of zwx woman lora:ghiles_3500:1" and this is what it gives me

00003-626981190

I try some of the saved lora files that it made during training. Same issue.

Steps to reproduce the problem

Train lora. Use lora.

Commit and libraries

Installed version of pytorch_optimizer: 2.12.0 [Dreambooth] pytorch_optimizer v2.12.0 is already installed. Installed version of Pillow: 9.5.0 [Dreambooth] Pillow is already installed. Installed version of tqdm: 4.66.1 [Dreambooth] tqdm is already installed. Installed version of tomesd: 0.1.3 [Dreambooth] tomesd v0.1.2 is already installed. Installed version of tensorboard: 2.13.0 [Dreambooth] tensorboard v2.13.0 is already installed. [+] torch version 2.0.1+cu118 installed. [+] torchvision version 0.15.2+cu118 installed. [+] accelerate version 0.21.0 installed. [+] diffusers version 0.25.0 installed. [+] bitsandbytes version 0.41.2.post2 installed. [!] xformers NOT installed. Launching Web UI with arguments: bin C:\stable-diffusion-webui-master\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.dll no module 'xformers'. Processing without... no module 'xformers'. Processing without... No module 'xformers'. Proceeding without it.

Command Line Arguments

No

Console logs

venv "C:\stable-diffusion-webui-master\venv\Scripts\Python.exe"
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: 1.7.0
Commit hash: <none>
Installing requirements
bin C:\stable-diffusion-webui-master\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.dll
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
If submitting an issue on github, please provide the full startup log for debugging purposes.

Initializing Dreambooth
Dreambooth revision: 71c3465b6c866050b147c58e2caf41984df2cf45
Checking xformers...
Checking bitsandbytes...
Checking bitsandbytes (Windows)
Virtual environment path: C:\stable-diffusion-webui-master\venv
Checking for C:\stable-diffusion-webui-master\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda111.dll
Found windows BNB DLL C:\stable-diffusion-webui-master\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda111.dll
Checking Dreambooth requirements...
Installed version of accelerate: 0.21.0
[Dreambooth] accelerate v0.21.0 is already installed.
Installed version of dadaptation: 3.2
[Dreambooth] dadaptation v3.2 is already installed.
Installed version of diffusers: 0.25.0
[Dreambooth] diffusers v0.25.0 is already installed.
Installed version of discord-webhook: 1.3.0
[Dreambooth] discord-webhook v1.3.0 is already installed.
Installed version of fastapi: 0.94.0
[Dreambooth] fastapi is already installed.
Installed version of gitpython: 3.1.32
[Dreambooth] gitpython v3.1.40 is not installed.
Successfully installed gitpython-3.1.41

Installed version of pytorch_optimizer: 2.12.0
[Dreambooth] pytorch_optimizer v2.12.0 is already installed.
Installed version of Pillow: 9.5.0
[Dreambooth] Pillow is already installed.
Installed version of tqdm: 4.66.1
[Dreambooth] tqdm is already installed.
Installed version of tomesd: 0.1.3
[Dreambooth] tomesd v0.1.2 is already installed.
Installed version of tensorboard: 2.13.0
[Dreambooth] tensorboard v2.13.0 is already installed.
[+] torch version 2.0.1+cu118 installed.
[+] torchvision version 0.15.2+cu118 installed.
[+] accelerate version 0.21.0 installed.
[+] diffusers version 0.25.0 installed.
[+] bitsandbytes version 0.41.2.post2 installed.
[!] xformers NOT installed.
Launching Web UI with arguments:
bin C:\stable-diffusion-webui-master\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.dll
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
Style database not found: C:\stable-diffusion-webui-master\styles.csv
Loading weights [6ce0161689] from C:\stable-diffusion-webui-master\models\Stable-diffusion\v1-5-pruned-emaonly.safetensors
Creating model from config: C:\stable-diffusion-webui-master\configs\v1-inference.yaml
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 16.5s (prepare environment: 10.1s, import torch: 2.3s, import gradio: 0.7s, setup paths: 0.8s, initialize shared: 0.2s, other imports: 0.3s, load scripts: 1.1s, create ui: 0.6s, gradio launch: 0.3s).
Advanced elements visible: False
Applying attention optimization: Doggettx... done.
Model loaded in 3.0s (load weights from disk: 0.4s, create model: 0.5s, apply weights to model: 1.8s, apply half(): 0.1s, calculate empty prompt: 0.1s).
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 10.33it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 12.69it/s]
Extracting config from C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\dreambooth\..\configs\v1-training-unfrozen.yaml
Extracting checkpoint from C:\stable-diffusion-webui-master\models\Stable-diffusion\v1-5-pruned-emaonly.safetensors
C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Duration: 00:00:10
Updating scheduler name to: DDIM
Advanced elements visible: True
Custom model name is ghiles
Initializing dreambooth training...
                                                                                                                       C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Init dataset!set:   0%|                                                                           | 0/1 [00:00<?, ?it/s]
Preparing Dataset (With Caching)
Bucket 0 (512, 512, 0) - Instance Images: 65 | Class Images: 0 | Max Examples/batch:  65
                                                                                                                       Saving cache!ed latents...: 100%|███████████████████████████████████████████████████████| 65/65 [00:03<00:00, 23.30it/s]
Total Buckets 1 - Instance Images: 65 | Class Images: 0 | Max Examples/batch: 65

Total images / batch: 65, total examples: 65███████████████████████████████████████████| 65/65 [00:03<00:00, 23.30it/s]
                  Initializing bucket counter!
Steps:   0%|                                                                                  | 0/3250 [00:00<?, ?it/s]C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\models\attention_processor.py:1946: FutureWarning: `LoRAAttnProcessor2_0` is deprecated and will be removed in version 0.26.0. Make sure use AttnProcessor2_0 instead by settingLoRA layers to `self.{to_q,to_k,to_v,to_out[0]}.lora_layer` respectively. This will be done automatically when using `LoraLoaderMixin.load_lora_weights`
  deprecate(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.28it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.61it/s]
Saving Lora Weights...:   0%|                                                                    | 0/1 [00:00<?, ?it/s]Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:14<00:28, 14.50s/it]
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  22%|████████▌                               | 700/3250 [01:42<04:46,  8.91it/s, loss=0.23, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.34it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.59it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.01s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  32%|████████████▎                         | 1050/3250 [02:40<04:08,  8.85it/s, loss=0.106, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.65it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.53it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.22s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  43%|████████████████▊                      | 1400/3250 [03:39<03:22,  9.12it/s, loss=0.24, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.27it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.48it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.28s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  54%|████████████████████▍                 | 1750/3250 [04:37<02:54,  8.62it/s, loss=0.123, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:01<00:00,  6.90it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.52it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.12s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  65%|████████████████████████▌             | 2100/3250 [05:35<02:11,  8.75it/s, loss=0.148, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.11it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.55it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.23s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  75%|███████████████████████████▉         | 2450/3250 [06:34<01:29,  8.89it/s, loss=0.0861, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:01<00:00,  6.90it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.50it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.42s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  86%|████████████████████████████████▋     | 2800/3250 [07:33<00:49,  9.04it/s, loss=0.168, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.57it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.61it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.11s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  97%|███████████████████████████████████▊ | 3150/3250 [08:30<00:10,  9.10it/s, loss=0.0699, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.48it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.56it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.35s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps: : 3500it [09:28,  9.15it/s, loss=0.0461, lr=0.0001, vram=5.8]                                                   C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.55it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.61it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.33s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
                                                                                                                       Cleanup completed.                                                                                | 0/3 [00:00<?, ?it/s]
Steps: : 3500it [09:48,  5.95it/s, loss=0.0461, lr=0.0001, vram=5.8]
  0%|                                                                                            | 0/3 [00:00<?, ?it/s]
Duration: 00:09:55
Duration: 00:09:56
loading network C:\stable-diffusion-webui-master\models\Lora\ghiles_3500.safetensors: AssertionError
Traceback (most recent call last):
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 279, in load_networks
    net = load_network(name, network_on_disk)
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 223, in load_network
    raise AssertionError(f"Could not find a module type (out of {', '.join([x.__class__.__name__ for x in module_types])}) that would accept those keys: {', '.join(weights.w)}")
AssertionError: Could not find a module type (out of ModuleTypeLora, ModuleTypeHada, ModuleTypeIa3, ModuleTypeLokr, ModuleTypeFull, ModuleTypeNorm, ModuleTypeGLora, ModuleTypeOFT) that would accept those keys: 0.attn1.to_k.lora_down.weight, 0.attn1.to_k.lora_up.weight, 0.attn1.to_out.0.lora_down.weight, 0.attn1.to_out.0.lora_up.weight, 0.attn1.to_q.lora_down.weight, 0.attn1.to_q.lora_up.weight, 0.attn1.to_v.lora_down.weight, 0.attn1.to_v.lora_up.weight, 0.attn2.to_k.lora_down.weight, 0.attn2.to_k.lora_up.weight, 0.attn2.to_out.0.lora_down.weight, 0.attn2.to_out.0.lora_up.weight, 0.attn2.to_q.lora_down.weight, 0.attn2.to_q.lora_up.weight, 0.attn2.to_v.lora_down.weight, 0.attn2.to_v.lora_up.weight

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 11.43it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 12.31it/s]
loading network C:\stable-diffusion-webui-master\models\Lora\ghiles_3500.safetensors: AssertionError1<00:00, 12.51it/s]
Traceback (most recent call last):
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 279, in load_networks
    net = load_network(name, network_on_disk)
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 223, in load_network
    raise AssertionError(f"Could not find a module type (out of {', '.join([x.__class__.__name__ for x in module_types])}) that would accept those keys: {', '.join(weights.w)}")
AssertionError: Could not find a module type (out of ModuleTypeLora, ModuleTypeHada, ModuleTypeIa3, ModuleTypeLokr, ModuleTypeFull, ModuleTypeNorm, ModuleTypeGLora, ModuleTypeOFT) that would accept those keys: 0.attn1.to_k.lora_down.weight, 0.attn1.to_k.lora_up.weight, 0.attn1.to_out.0.lora_down.weight, 0.attn1.to_out.0.lora_up.weight, 0.attn1.to_q.lora_down.weight, 0.attn1.to_q.lora_up.weight, 0.attn1.to_v.lora_down.weight, 0.attn1.to_v.lora_up.weight, 0.attn2.to_k.lora_down.weight, 0.attn2.to_k.lora_up.weight, 0.attn2.to_out.0.lora_down.weight, 0.attn2.to_out.0.lora_up.weight, 0.attn2.to_q.lora_down.weight, 0.attn2.to_q.lora_up.weight, 0.attn2.to_v.lora_down.weight, 0.attn2.to_v.lora_up.weight

100%|██████████████████████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 10.50it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 10.71it/s]
loading network C:\stable-diffusion-webui-master\models\Lora\ghiles_3150.safetensors: AssertionError3<00:00, 10.71it/s]
Traceback (most recent call last):
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 279, in load_networks
    net = load_network(name, network_on_disk)
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 223, in load_network
    raise AssertionError(f"Could not find a module type (out of {', '.join([x.__class__.__name__ for x in module_types])}) that would accept those keys: {', '.join(weights.w)}")
AssertionError: Could not find a module type (out of ModuleTypeLora, ModuleTypeHada, ModuleTypeIa3, ModuleTypeLokr, ModuleTypeFull, ModuleTypeNorm, ModuleTypeGLora, ModuleTypeOFT) that would accept those keys: 0.attn1.to_k.lora_down.weight, 0.attn1.to_k.lora_up.weight, 0.attn1.to_out.0.lora_down.weight, 0.attn1.to_out.0.lora_up.weight, 0.attn1.to_q.lora_down.weight, 0.attn1.to_q.lora_up.weight, 0.attn1.to_v.lora_down.weight, 0.attn1.to_v.lora_up.weight, 0.attn2.to_k.lora_down.weight, 0.attn2.to_k.lora_up.weight, 0.attn2.to_out.0.lora_down.weight, 0.attn2.to_out.0.lora_up.weight, 0.attn2.to_q.lora_down.weight, 0.attn2.to_q.lora_up.weight, 0.attn2.to_v.lora_down.weight, 0.attn2.to_v.lora_up.weight

100%|██████████████████████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 11.28it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 11.02it/s]
loading network C:\stable-diffusion-webui-master\models\Lora\ghiles_2800.safetensors: AssertionError3<00:00, 11.78it/s]
Traceback (most recent call last):
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 279, in load_networks
    net = load_network(name, network_on_disk)
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 223, in load_network
    raise AssertionError(f"Could not find a module type (out of {', '.join([x.__class__.__name__ for x in module_types])}) that would accept those keys: {', '.join(weights.w)}")
AssertionError: Could not find a module type (out of ModuleTypeLora, ModuleTypeHada, ModuleTypeIa3, ModuleTypeLokr, ModuleTypeFull, ModuleTypeNorm, ModuleTypeGLora, ModuleTypeOFT) that would accept those keys: 0.attn1.to_k.lora_down.weight, 0.attn1.to_k.lora_up.weight, 0.attn1.to_out.0.lora_down.weight, 0.attn1.to_out.0.lora_up.weight, 0.attn1.to_q.lora_down.weight, 0.attn1.to_q.lora_up.weight, 0.attn1.to_v.lora_down.weight, 0.attn1.to_v.lora_up.weight, 0.attn2.to_k.lora_down.weight, 0.attn2.to_k.lora_up.weight, 0.attn2.to_out.0.lora_down.weight, 0.attn2.to_out.0.lora_up.weight, 0.attn2.to_q.lora_down.weight, 0.attn2.to_q.lora_up.weight, 0.attn2.to_v.lora_down.weight, 0.attn2.to_v.lora_up.weight

100%|██████████████████████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 11.14it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 11.45it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 11.56it/s]

Additional information

No response

klubkelli commented 5 months ago

GitPython==3.1.32 Pillow==9.5.0 accelerate==0.21.0 basicsr==1.4.2 blendmodes==2022 clean-fid==0.1.35 einops==0.4.1 fastapi==0.94.0 gfpgan==1.3.8 gradio==3.41.2 httpcore==0.15 inflection==0.5.1 jsonmerge==1.8.0 kornia==0.6.7 lark==1.1.2 numpy==1.23.5 omegaconf==2.2.3 open-clip-torch==2.20.0 piexif==1.1.3 psutil==5.9.5 pytorch_lightning==1.9.4 realesrgan==0.3.0 resize-right==0.0.2 safetensors==0.3.1 scikit-image==0.21.0 timm==0.9.2 tomesd==0.1.3 torch torchdiffeq==0.2.3 torchsde==0.2.6 transformers==4.35.2 httpx==0.24.1

klubkelli commented 5 months ago

Checking Dreambooth requirements... Installed version of accelerate: 0.21.0 [Dreambooth] accelerate v0.21.0 is already installed. Installed version of dadaptation: 3.2 [Dreambooth] dadaptation v3.2 is already installed. Installed version of diffusers: 0.25.0 [Dreambooth] diffusers v0.25.0 is already installed. Installed version of discord-webhook: 1.3.0 [Dreambooth] discord-webhook v1.3.0 is already installed. Installed version of fastapi: 0.94.0 [Dreambooth] fastapi is already installed. Installed version of gitpython: 3.1.32 [Dreambooth] gitpython v3.1.40 is not installed. Successfully installed gitpython-3.1.41

Installed version of pytorch_optimizer: 2.12.0 [Dreambooth] pytorch_optimizer v2.12.0 is already installed. Installed version of Pillow: 9.5.0 [Dreambooth] Pillow is already installed. Installed version of tqdm: 4.66.1 [Dreambooth] tqdm is already installed. Installed version of tomesd: 0.1.3 [Dreambooth] tomesd v0.1.2 is already installed. Installed version of tensorboard: 2.13.0 [Dreambooth] tensorboard v2.13.0 is already installed. [+] torch version 2.0.1+cu118 installed. [+] torchvision version 0.15.2+cu118 installed. [+] accelerate version 0.21.0 installed. [+] diffusers version 0.25.0 installed. [+] bitsandbytes version 0.41.2.post2 installed. [!] xformers NOT installed.

klubkelli commented 5 months ago

Any suggestions?

klubkelli commented 5 months ago

version: [v1.7.0] python: 3.10.6 torch: 2.0.1+cu118 xformers: N/A gradio: 3.41.2 checkpoint: [6ce0161689]

github-actions[bot] commented 4 months ago

This issue is stale because it has been open for 14 days with no activity. Remove stale label or comment or this will be closed in 30 days

Mikhael-Danilov commented 4 months ago

Same issue when using 731da11f9c800ac39e0a9a668d716ce76a151453 commit.

NutritiousCookie commented 4 months ago

Having similar issues, but I'll point you towards #857 #892 #897 which seem to cover this issue as well

github-actions[bot] commented 3 months ago

This issue is stale because it has been open for 14 days with no activity. Remove stale label or comment or this will be closed in 30 days

eija06 commented 3 months ago

this is issue https://github.com/d8ahazard/sd_dreambooth_extension/issues/1439 not the ones mentioned above

github-actions[bot] commented 2 months ago

This issue is stale because it has been open for 14 days with no activity. Remove stale label or comment or this will be closed in 30 days

klubkelli commented 2 months ago

This issue is still happening. Has anyone ever figured out how to fix this Dreambooth Lora bug?

eija06 commented 2 months ago

This issue is still happening. Has anyone ever figured out how to fix this Dreambooth Lora bug?

yes, but only by manually replacing the incorrect LORA keys mentioned in the error message to the correct ones, using a sed script on the output file. you can find the translation of the keys in this file: https://gist.github.com/eavae/a336029601f726737a0fd070f44c2f8a

But this is not a solution and I am getting no replies in #1439. My advice is to abandon this extension and use actively maintained software.

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 14 days with no activity. Remove stale label or comment or this will be closed in 30 days

d8ahazard / sd_dreambooth_extension