search

ljleb / comfy-mecha

Executable Stable Diffusion merge recipes in comfyui
MIT License
58 stars 3 forks source link

Increase compatibility with LyCORIS? #18

Open And233 opened 3 weeks ago

And233 commented 3 weeks ago

it seems that it can not load lokr with Lora Mecha Recipe, since they have different layer name.

And233 commented 3 weeks ago

oh, i miss the new-config branch of sd-mecha. So it will be compatible soon?

ljleb commented 2 weeks ago

I don't know when I'll be able to finish the config update v0.1.0. Indeed, it contains many improvements to the library.

It's been long overdue. I have ran into situations I did not initially account for, and it became this huge upcoming update as a result. I have very recently found a good way to support model conversion (i.e. SDXL lyco to SGM, or SGM to diffusers, or flux-backbone-only to flux-with-text-encoder, etc.) which has been a huge problem for a while. Although this is more or less in the past now, the API and implementation still need more work.

With the structure in v0.0.X (prior to this upcoming update), it is not possible (or very impractical) to support some kinds of lycoris, for example BOFT. With the new update it will become practical and hopefully straightforward to implement loading BOFT for all architectures.

The major problem for me right now is understanding how and why keys differ between different implementations to avoid duplicating the same conversion code in 20 places, especially for loras. It is a huge mess. I also need to register model configs from kohya to generate the appropriate keys for lycoris models.

Additionally, I'm not able to spend as much time as I would like maintaining this library. I am doing a lot of experiments on the side, and real life also gets in the way.

Once this update is done, I will focus on writing docs. I suppose it could make it easier for potential contributors to lend a hand, and at the very least it would make the lib less obscure to learn to use.