search

ogkalu2 / Merge-Stable-Diffusion-models-without-distortion

Adaptation of the merging method described in the paper - Git Re-Basin: Merging Models modulo Permutation Symmetries (https://arxiv.org/abs/2209.04836) for Stable Diffusion
MIT License
139 stars 21 forks source link

KeyError: 'embedding_manager.embedder.transformer.text_model.embeddings.position_ids' #15

Open D-Ogi opened 1 year ago

D-Ogi commented 1 year ago

I got an error when running python SD_rebasin_merge.py --model_a A111merge.ckpt --model_b DBpruned.ckpt" --device cpu --iterations 20 --alpha 0.6

    ---------------------
         ITERATION 1
    ---------------------

new alpha = 0.03

FINDING PERMUTATIONS
P_bg337: -0.5
P_bg358: -0.0625
Traceback (most recent call last):
  File "P:\a1111\sd-basin\Merge-Stable-Diffusion-models-without-distortion\SD_rebasin_merge.py", line 68, in <module>
    theta_0 = apply_permutation(permutation_spec, first_permutation, theta_0)
  File "P:\a1111\sd-basin\Merge-Stable-Diffusion-models-without-distortion\weight_matching.py", line 784, in apply_permutation
    return {k: get_permuted_param(ps, perm, k, params) for k in params.keys()}
  File "P:\a1111\sd-basin\Merge-Stable-Diffusion-models-without-distortion\weight_matching.py", line 784, in <dictcomp>
    return {k: get_permuted_param(ps, perm, k, params) for k in params.keys()}
  File "P:\a1111\sd-basin\Merge-Stable-Diffusion-models-without-distortion\weight_matching.py", line 771, in get_permuted_param
    for axis, p in enumerate(ps.axes_to_perm[k]):
KeyError: 'embedding_manager.embedder.transformer.text_model.embeddings.position_ids'

Any ideas how to fix it?

ogkalu2 commented 1 year ago

Skip it in the permutation spec. There's a list of skipped layers in there. Just add this one to it.

so **skip("embedding_manager.embedder.transformer.text_model.embeddings.position_ids", None, None)

D-Ogi commented 1 year ago

It passed that line, but now I get same errors are about these layers. I guess I should do some dense and norm operations with them, right?

embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.k_proj.weight
embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.k_proj.bias
embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.v_proj.weight
embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.v_proj.bias
embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.q_proj.weight
embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.q_proj.bias
D-Ogi commented 1 year ago

This is how far I went to, but still new layers appear...

     **skip("embedding_manager.embedder.transformer.text_model.embeddings.position_ids", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.embeddings.token_embedding.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.embeddings.position_embedding.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.k_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.k_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.v_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.v_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.q_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.q_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.out_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.self_attn.out_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.layer_norm1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.layer_norm1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.mlp.fc1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.mlp.fc1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.mlp.fc2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.mlp.fc2.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.layer_norm2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.0.layer_norm2.bias", None, None),

     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.self_attn.k_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.self_attn.k_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.self_attn.v_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.self_attn.v_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.self_attn.q_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.self_attn.q_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.self_attn.out_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.self_attn.out_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.layer_norm1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.layer_norm1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.mlp.fc1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.mlp.fc1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.mlp.fc2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.mlp.fc2.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.layer_norm2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.1.layer_norm2.bias", None, None),

     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.self_attn.k_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.self_attn.k_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.self_attn.v_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.self_attn.v_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.self_attn.q_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.self_attn.q_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.self_attn.out_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.self_attn.out_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.layer_norm1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.layer_norm1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.mlp.fc1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.mlp.fc1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.mlp.fc2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.mlp.fc2.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.layer_norm2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.2.layer_norm2.bias", None, None),

     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.self_attn.k_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.self_attn.k_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.self_attn.v_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.self_attn.v_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.self_attn.q_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.self_attn.q_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.self_attn.out_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.self_attn.out_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.layer_norm1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.layer_norm1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.mlp.fc1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.mlp.fc1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.mlp.fc2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.mlp.fc2.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.layer_norm2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.3.layer_norm2.bias", None, None),

     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.self_attn.k_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.self_attn.k_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.self_attn.v_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.self_attn.v_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.self_attn.q_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.self_attn.q_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.self_attn.out_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.self_attn.out_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.layer_norm1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.layer_norm1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.mlp.fc1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.mlp.fc1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.mlp.fc2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.mlp.fc2.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.layer_norm2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.4.layer_norm2.bias", None, None),

     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.self_attn.k_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.self_attn.k_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.self_attn.v_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.self_attn.v_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.self_attn.q_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.self_attn.q_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.self_attn.out_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.self_attn.out_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.layer_norm1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.layer_norm1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.mlp.fc1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.mlp.fc1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.mlp.fc2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.mlp.fc2.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.layer_norm2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.5.layer_norm2.bias", None, None),

     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.self_attn.k_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.self_attn.k_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.self_attn.v_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.self_attn.v_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.self_attn.q_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.self_attn.q_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.self_attn.out_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.self_attn.out_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.layer_norm1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.layer_norm1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.mlp.fc1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.mlp.fc1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.mlp.fc2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.mlp.fc2.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.layer_norm2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.6.layer_norm2.bias", None, None),

     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.self_attn.k_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.self_attn.k_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.self_attn.v_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.self_attn.v_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.self_attn.q_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.self_attn.q_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.self_attn.out_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.self_attn.out_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.layer_norm1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.layer_norm1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.mlp.fc1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.mlp.fc1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.mlp.fc2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.mlp.fc2.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.layer_norm2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.7.layer_norm2.bias", None, None),

     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.self_attn.k_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.self_attn.k_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.self_attn.v_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.self_attn.v_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.self_attn.q_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.self_attn.q_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.self_attn.out_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.self_attn.out_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.layer_norm1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.layer_norm1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.mlp.fc1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.mlp.fc1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.mlp.fc2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.mlp.fc2.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.layer_norm2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.8.layer_norm2.bias", None, None),

     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.self_attn.k_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.self_attn.k_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.self_attn.v_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.self_attn.v_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.self_attn.q_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.self_attn.q_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.self_attn.out_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.self_attn.out_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.layer_norm1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.layer_norm1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.mlp.fc1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.mlp.fc1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.mlp.fc2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.mlp.fc2.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.layer_norm2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.9.layer_norm2.bias", None, None),

     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.self_attn.k_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.self_attn.k_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.self_attn.v_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.self_attn.v_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.self_attn.q_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.self_attn.q_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.self_attn.out_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.self_attn.out_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.layer_norm1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.layer_norm1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.mlp.fc1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.mlp.fc1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.mlp.fc2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.mlp.fc2.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.layer_norm2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.10.layer_norm2.bias", None, None),

     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.self_attn.k_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.self_attn.k_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.self_attn.v_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.self_attn.v_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.self_attn.q_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.self_attn.q_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.self_attn.out_proj.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.self_attn.out_proj.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.layer_norm1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.layer_norm1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.mlp.fc1.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.mlp.fc1.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.mlp.fc2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.mlp.fc2.bias", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.layer_norm2.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.encoder.layers.11.layer_norm2.bias", None, None),

     **skip("embedding_manager.embedder.transformer.text_model.final_layer_norm.weight", None, None),
     **skip("embedding_manager.embedder.transformer.text_model.final_layer_norm.bias", None, None),
     **skip("model_ema.diffusion_modeltime_embed0weight", None, None),
     **skip("model_ema.diffusion_modeltime_embed0bias", None, None),
     **skip("model_ema.diffusion_modeltime_embed2weight", None, None),
     **skip("model_ema.diffusion_modeltime_embed2bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks00weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks10in_layers0weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks10in_layers0bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks00bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks10in_layers2weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks10in_layers2bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks10emb_layers1weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks10emb_layers1bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks10out_layers0weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks10out_layers0bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks10out_layers1weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks10out_layers1bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks10out_layers3weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks10out_layers3bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11normweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11normbias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11proj_inweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11proj_inbias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0attn1to_qweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0attn1to_kweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0attn1to_vweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0attn1to_out0weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0attn1to_out0bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0ffnet0projweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0ffnet2projweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0ffnet2weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0ffnet0projbias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0ffnet2projbias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0ffnet2bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0attn2to_qweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0attn2to_kweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0attn2to_vweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0attn2to_out0weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0attn2to_out0bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0norm1weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0norm1bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0norm2weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0norm2bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0norm3weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11transformer_blocks0norm3bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11proj_outweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks11proj_outbias", None, None),

     **skip("model_ema.diffusion_modelinput_blocks40in_layers0weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40in_layers0bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40in_layers2weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40in_layers2bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40emb_layers1weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40emb_layers1bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40out_layers0weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40out_layers0bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40out_layers1weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40out_layers1bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40out_layers3weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40out_layers3bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21normweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21normbias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21proj_inweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21proj_inbias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0attn1to_qweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0attn1to_kweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0attn1to_vweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0attn1to_out0weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0attn1to_out0bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0ffnet0projweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0ffnet2projweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0ffnet2weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0ffnet0projbias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0ffnet2projbias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0ffnet2bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0attn2to_qweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0attn2to_kweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0attn2to_vweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0attn2to_out0weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0attn2to_out0bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0norm1weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0norm1bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0norm2weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0norm2bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0norm3weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21transformer_blocks0norm3bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21proj_outweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks21proj_outbias", None, None),

     **skip("model_ema.diffusion_modelinput_blocks30opweight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks30opbias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40in_layers0weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40in_layers0bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40in_layers2weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40in_layers2bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40emb_layers1weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40emb_layers1bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40out_layers0weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40out_layers0bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40out_layers1weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40out_layers1bias", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40out_layers3weight", None, None),
     **skip("model_ema.diffusion_modelinput_blocks40out_layers3bias", None, None),
D-Ogi commented 1 year ago

Now I get a following error:

Using half precision

    ---------------------
         ITERATION 1
    ---------------------

new alpha = 0.025

FINDING PERMUTATIONS
P_bg358: -0.1875
P_bg337: -1.0
Traceback (most recent call last):
  File "P:\a1111\sd-basin\Merge-Stable-Diffusion-models-without-distortion\SD_rebasin_merge.py", line 68, in <module>
    theta_0 = apply_permutation(permutation_spec, first_permutation, theta_0)
  File "P:\a1111\sd-basin\Merge-Stable-Diffusion-models-without-distortion\weight_matching.py", line 1099, in apply_permutation
    return {k: get_permuted_param(ps, perm, k, params) for k in params.keys()}
  File "P:\a1111\sd-basin\Merge-Stable-Diffusion-models-without-distortion\weight_matching.py", line 1099, in <dictcomp>
    return {k: get_permuted_param(ps, perm, k, params) for k in params.keys()}
  File "P:\a1111\sd-basin\Merge-Stable-Diffusion-models-without-distortion\weight_matching.py", line 1093, in get_permuted_param
    w = torch.index_select(w, axis, perm[p].int())
RuntimeError: INDICES element is out of DATA bounds, id=4 axis_dim=4
ogkalu2 commented 1 year ago

Dude... What the hell are you trying to merge. Which models ?

D-Ogi commented 1 year ago

It's sdhassan 1.4.