LarryJane491 / Lora-Training-in-Comfy

This custom node lets you train LoRA directly in ComfyUI!
388 stars 57 forks source link

raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( #74

Open andreclaudino opened 2 months ago

andreclaudino commented 2 months ago

I don't know whats is happening, when trying to train it raises the following error, and a few seconds later, says training was concluded (of course, there is no lora trained and it failed).

[Dataset 0]
loading image sizes.
100%|██████████| 294/294 [00:00<00:00, 23123.85it/s]
/home/user/miniconda3/envs/ComfyUI/lib/python3.12/site-packages/accelerate/accelerator.py:494: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
  self.scaler = torch.cuda.amp.GradScaler(**kwargs)
make buckets
number of images (including repeats) / 各bucketの画像枚数(繰り返し回数を含む)
bucket 0: resolution (320, 704), count: 89082
mean ar error (without repeats): 0.04545454545454546
preparing accelerator
loading model for process 0/1
load StableDiffusion checkpoint: /home/user/Aplicacoes/ComfyUI/models/checkpoints/SD15/realDream_turboLCM6.safetensors
UNet2DConditionModel: 64, [5, 10, 20, 20], 1024, False, False
Traceback (most recent call last):
  File "/home/user/Aplicacoes/ComfyUI/custom_nodes/Lora-Training-in-Comfy/sd-scripts/train_network.py", line 1012, in <module>
    trainer.train(args)
  File "/home/user/Aplicacoes/ComfyUI/custom_nodes/Lora-Training-in-Comfy/sd-scripts/train_network.py", line 228, in train
    model_version, text_encoder, vae, unet = self.load_target_model(args, weight_dtype, accelerator)
                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Aplicacoes/ComfyUI/custom_nodes/Lora-Training-in-Comfy/sd-scripts/train_network.py", line 102, in load_target_model
    text_encoder, vae, unet, _ = train_util.load_target_model(args, weight_dtype, accelerator)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Aplicacoes/ComfyUI/custom_nodes/Lora-Training-in-Comfy/sd-scripts/library/train_util.py", line 3917, in load_target_model
    text_encoder, vae, unet, load_stable_diffusion_format = _load_target_model(
                                                            ^^^^^^^^^^^^^^^^^^^
  File "/home/user/Aplicacoes/ComfyUI/custom_nodes/Lora-Training-in-Comfy/sd-scripts/library/train_util.py", line 3860, in _load_target_model
    text_encoder, vae, unet = model_util.load_models_from_stable_diffusion_checkpoint(
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Aplicacoes/ComfyUI/custom_nodes/Lora-Training-in-Comfy/sd-scripts/library/model_util.py", line 1007, in load_models_from_stable_diffusion_checkpoint
    info = unet.load_state_dict(converted_unet_checkpoint)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/miniconda3/envs/ComfyUI/lib/python3.12/site-packages/torch/nn/modules/module.py", line 2215, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for UNet2DConditionModel:

Any suggestions?

FahimulBari commented 2 months ago

Same Problem. Whatever I do it is giving the same errors!

[Dataset 0] loading image sizes. 100%|███████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 859.39it/s] make buckets number of images (including repeats) / 各bucketの画像枚数(繰り返し回数を含む) bucket 0: resolution (512, 512), count: 60 mean ar error (without repeats): 0.0 preparing accelerator C:\Users\Fahimul Bari\AppData\Local\Programs\Python\Python310\lib\site-packages\accelerate\accelerator.py:437: FutureWarning: torch.cuda.amp.GradScaler(args...) is deprecated. Please use torch.amp.GradScaler('cuda', args...) instead. self.scaler = torch.cuda.amp.GradScaler(**kwargs) loading model for process 0/1 load StableDiffusion checkpoint: G:\@ComfyUI\ComfyUI_windows_portable\ComfyUI\models\checkpoints\SD1.5\dreamshaper_8.safetensors UNet2DConditionModel: 64, 8, 768, False, False loading u-net: loading vae: Traceback (most recent call last): File "G:\@ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\Lora-Training-in-Comfy\sd-scripts\train_network.py", line 1012, in trainer.train(args) File "G:\@ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\Lora-Training-in-Comfy\sd-scripts\train_network.py", line 228, in train model_version, text_encoder, vae, unet = self.load_target_model(args, weight_dtype, accelerator) File "G:\@ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\Lora-Training-in-Comfy\sd-scripts\train_network.py", line 102, in load_target_model textencoder, vae, unet, = train_util.load_target_model(args, weight_dtype, accelerator) File "G:\@ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\Lora-Training-in-Comfy\sd-scripts\library\train_util.py", line 3917, in load_target_model text_encoder, vae, unet, load_stable_diffusion_format = _load_target_model( File "G:\@ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\Lora-Training-in-Comfy\sd-scripts\library\train_util.py", line 3860, in _load_target_model text_encoder, vae, unet = model_util.load_models_from_stable_diffusion_checkpoint( File "G:\@ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\Lora-Training-in-Comfy\sd-scripts\library\model_util.py", line 1072, in load_models_from_stable_diffusion_checkpoint info = text_model.load_state_dict(converted_text_encoder_checkpoint) File "C:\Users\Fahimul Bari\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 2215, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for CLIPTextModel: Unexpected key(s) in state_dict: "text_model.embeddings.position_ids".