search

ogkalu2 / Merge-Stable-Diffusion-models-without-distortion

Adaptation of the merging method described in the paper - Git Re-Basin: Merging Models modulo Permutation Symmetries (https://arxiv.org/abs/2209.04836) for Stable Diffusion
MIT License
139 stars 21 forks source link

SD 2.x model merge broken (or just not implemented yet) #38

Open gxmikvid opened 1 year ago

gxmikvid commented 1 year ago

First of all, great work, used this back then to screw around with models.

Got back into SD, tried to merge 2.x models, got this error: KeyError: 'cond_stage_model.model.transformer.text_model.embeddings.position_ids' Any 2.x model does this! Removed ema as per this issue.

Tried some of the fixes proposed. This was the closest to my problem, proposed solutions did nothing, i assumed the code was not made to handle SD 2.x. Perhaps it has a different/no clip model. There was an issue that detailed it but can't find it at the moment.

Are you planning on supporting 2.x models any time soon? There was an issue that asked a similar question but about 1.x-2.x cross merging, and i feel like my request is different enough.

Love this repo. No rush.

ogkalu2 commented 1 year ago

@gxmikvid

Hi. Merging 2.x models can be done but it's something I'd have to support in the code first. To be honest, I didn't really have any particular plans to support it since the models seem to be used much less than 1.5 but I'll see what I can do.

gxmikvid commented 1 year ago

Thank you so much. Also, I saw your answer to this too late, so my question might have been redundant. Sorry for that, my bad.

6DammK9 commented 4 months ago

Write the permutation spec dedicate for the SD2.X will be good. I may have a try since SD2 is a subset of SDXL, but the layer name is different Also I need some more trick to detect a SD2.X model.

Edit: PR has been made