search

maszhongming / Multi-LoRA-Composition

Repository for the Paper "Multi-LoRA Composition for Image Generation"
422 stars 45 forks source link

What about a spherical linear interpolation? #9

Open leepeee opened 4 months ago

leepeee commented 4 months ago

Hi! Composing the ``CFG style" noise is a great idea. I wonder if using a specific interpolation of two noises, instead of just taking the mean, can achieve better performance. What about the spherical linear interpolation mentioned in DDIM?

maszhongming commented 3 months ago

Thank you for your interest in our work!

We briefly experimented with some combinations of interpolated or dynamic weights to get the final score, but they didn't yield significant improvement, so we ended up using the average score in the paper. We haven't explored spherical linear interpolation yet, so that might be an interesting direction to explore. In addition, another flexible direction would be to combine LoRA-S and LoRA-C at the same time, e.g., activating a subset of all LoRAs at each step and applying a weighted sum (either through interpolation or dynamic weighting) to each LoRA in the subset, which could potentially enhance the results.