hako-mikan / sd-webui-supermerger

model merge extention for stable diffusion web ui
GNU Affero General Public License v3.0
735 stars 109 forks source link

Cannot extract LORA from trained model #359

Closed Bielecki closed 5 months ago

Bielecki commented 6 months ago

I have trained a few checkpoints with Dreambooth extension and I want to extract LORA from them. I have done that in the past (about a year ago) and it worked, but now I can't.

SD details:

Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.8.0
Commit hash: bef51aed032c0aaa5cfd80445bc4cf0d85b408b5

Error log:

make LoRA start
Loading weights [326840db30] from J:\stable-diffusion-webui\models\Stable-diffusion\mln2\mln2_38641.safetensors
Creating model from config: J:\stable-diffusion-webui\models\Stable-diffusion\mln2\mln2_38641.yaml
Applying attention optimization: xformers... done.
Model loaded in 9.1s (load weights from disk: 0.1s, create model: 0.4s, apply weights to model: 8.2s, load textual inversion embeddings: 0.2s).
Detected model type: SDXL: False, SD2.X: False, SD1.X: True
loading original SD model : J:\stable-diffusion-webui\models\Stable-diffusion\stable-diffusion-additional-models\picxReal_10.safetensors
UNet2DConditionModel: 64, 8, 768, False, False
loading u-net: <All keys matched successfully>
Traceback (most recent call last):
  File "J:\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "J:\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "J:\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "J:\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "J:\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "J:\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "J:\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "J:\stable-diffusion-webui\extensions\sd-webui-supermerger\scripts\mergers\pluslora.py", line 302, in makelora
    result = ext.svd(args)
  File "J:\stable-diffusion-webui\extensions\sd-webui-supermerger\scripts\kohyas\extract_lora_from_models.py", line 52, in svd
    text_encoder_o, _, unet_o = model_util.load_models_from_stable_diffusion_checkpoint(args.v2, args.model_org)
  File "J:\stable-diffusion-webui\extensions\sd-webui-supermerger\scripts\kohyas\model_util.py", line 1066, in load_models_from_stable_diffusion_checkpoint
    info = text_model.load_state_dict(converted_text_encoder_checkpoint)
  File "J:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for CLIPTextModel:
        Missing key(s) in state_dict: "text_model.embeddings.position_ids".

Source model: https://civitai.com/models/241415

tmasakas commented 6 months ago

Facing the same issue, tried on various models and on SD 1.6/1.7 to no avail

miasik commented 6 months ago

The same here web-ui 1.8

hako-mikan commented 6 months ago

Same to https://github.com/hako-mikan/sd-webui-supermerger/issues/329, fixed.