Request: Can the code be modified to merge SD 1.x and SD 2.1 checkpoints?

ogkalu2 / Merge-Stable-Diffusion-models-without-distortion

Adaptation of the merging method described in the paper - Git Re-Basin: Merging Models modulo Permutation Symmetries (https://arxiv.org/abs/2209.04836) for Stable Diffusion

MIT License

139 stars 21 forks source link

Request: Can the code be modified to merge SD 1.x and SD 2.1 checkpoints? #21

Open gessyoo opened 1 year ago

gessyoo commented 1 year ago

The code works very well with SD 1.x models, thanks. Can it be modified to re-basin existing SD 1.x checkpoints -- there are many -- with the new SD 2.1 models? I've tried but it throws an Runtime error pointing to line 55, that "the size of tensor a (768) must match the size of tensor b (1024) ,,,"

ogkalu2 commented 1 year ago

I can add the perm spec for SD 2 later but a bit busy right now. When I do, you should be able to merge multiple v2 models but merging v1 and v2 models together will not work

You can merge with re-basin by excluding done layers and I've tried that but it just spits out a model that produces colorful noise so far.