search

Jack000 / glid-3-xl-stable

stable diffusion training
MIT License
290 stars 36 forks source link

Merging Two Models #10

Open ExponentialML opened 1 year ago

ExponentialML commented 1 year ago

Hello, thanks for your implementation. I've scoured the internet for merging multiple LDMs into one model, but there doesn't seem to be a viable solution as of yet. Would it be possible to do so?

For example, you might have one model finetuned on scenery, and one on animals. Would merging two be a viable solution, or would some form of transfer learning be a better alternative? I've tried merging state_dict between two models, but it expands so many dimensions that I don't think it would work properly during inference.

Any insight would be greatly appreciated!

Jack000 commented 1 year ago

not sure but maybe have a look at https://github.com/energy-based-model/Compositional-Visual-Generation-with-Composable-Diffusion-Models-PyTorch