search

ogkalu2 / Merge-Stable-Diffusion-models-without-distortion

Adaptation of the merging method described in the paper - Git Re-Basin: Merging Models modulo Permutation Symmetries (https://arxiv.org/abs/2209.04836) for Stable Diffusion
MIT License
139 stars 21 forks source link

Are models being generated with the correct alpha? #24

Open brucethemoose opened 1 year ago

brucethemoose commented 1 year ago

With an alpha of 0.85 in the cli, my output looks like this:

    ---------------------
         ITERATION 1
    ---------------------

new alpha = 0.08499999999999999

FINDING PERMUTATIONS
P_bg337: -0.5
P_bg358: 0.0625
P_bg358: 0.875
P_bg337: -0.5

    ---------------------
         ITERATION 2
    ---------------------

new alpha = 0.09289617486338797

FINDING PERMUTATIONS
P_bg358: 0.3125
P_bg337: 0.5
P_bg358: 0.8125
P_bg324: 1.0

    ---------------------
         ITERATION 3
    ---------------------

new alpha = 0.10240963855421692

FINDING PERMUTATIONS
P_bg358: 0.3125
P_bg324: -0.5
P_bg358: 0.5625

    ---------------------
         ITERATION 4
    ---------------------

new alpha = 0.11409395973154357

FINDING PERMUTATIONS
P_bg324: 1.0
P_bg358: 0.3125
P_bg358: 0.5

    ---------------------
         ITERATION 5
    ---------------------

new alpha = 0.12878787878787867

FINDING PERMUTATIONS
P_bg337: 1.0
P_bg358: 0.3125
P_bg358: 0.5625
P_bg324: 0.5

    ---------------------
         ITERATION 6
    ---------------------

new alpha = 0.14782608695652188

FINDING PERMUTATIONS
P_bg358: 0.3125
P_bg337: -2.0
P_bg324: 0.5
P_bg324: 0.5
P_bg358: 0.375
P_bg337: 1.0

    ---------------------
         ITERATION 7
    ---------------------

new alpha = 0.173469387755102

FINDING PERMUTATIONS
P_bg337: 0.5
P_bg358: 0.0625
P_bg324: -0.5
P_bg358: 0.3125

    ---------------------
         ITERATION 8
    ---------------------

new alpha = 0.20987654320987648

FINDING PERMUTATIONS
P_bg358: 0.3125
P_bg337: 0.5
P_bg358: 0.375
P_bg324: -0.5

    ---------------------
         ITERATION 9
    ---------------------

new alpha = 0.2656249999999999

FINDING PERMUTATIONS
P_bg337: 0.5
P_bg358: 0.1875
P_bg358: 0.5

    ---------------------
         ITERATION 10
    ---------------------

new alpha = 0.3617021276595742

FINDING PERMUTATIONS
P_bg358: -0.125
P_bg358: 0.75

Saving...
Done!

So the last "new alpha" is... 0.36, which doesn't seem right.

Am I just misinterpreting the output? Do I need more iterations? It seems to behave similarly with lower alpha values as well, ending up way lower than the initial target on iteration 10.

brucethemoose commented 1 year ago

playing with some merged models more, it seems like its properly respecting the alpha value.