search

ogkalu2 / Merge-Stable-Diffusion-models-without-distortion

Adaptation of the merging method described in the paper - Git Re-Basin: Merging Models modulo Permutation Symmetries (https://arxiv.org/abs/2209.04836) for Stable Diffusion
MIT License
139 stars 21 forks source link

Merging breaks position_ids in CLIP model #29

Closed zwishenzug closed 1 year ago

zwishenzug commented 1 year ago

Even though the position_ids are supposed to be ignored I think the code still converts them to floats/halfs which they shouldn't be, they should be int64.

A quick fix I did was to put this line before the torch.save:

theta_0['cond_stage_model.transformer.text_model.embeddings.position_ids'] = torch.tensor([list(range(77))], dtype=torch.int64, device="cpu")