search

comfyanonymous / ComfyUI

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
https://www.comfy.org/
GNU General Public License v3.0
51.1k stars 5.37k forks source link

Is it possible to use different lora models in different regions? #374

Open KimigaiiWuyi opened 1 year ago

KimigaiiWuyi commented 1 year ago

I don't know if this is even possible... I noticed that in the last Ksampler node, the same model is always connected to that node. In the last step, it seems to be contaminated by the Lora model for the whole area composition?

Season02 commented 1 year ago

I have the exactly same question

pxjhll commented 1 year ago

I have the exactly same question +1

ShyRed commented 1 year ago

Same question. Just started messing around with regions in 1111 and using Composable Lora plugin you can use different LoRA per region.

Would like to know if that is possible with ComyUI, too. A small example would be awesome.

daizhuo commented 1 year ago

I tried multi laten composition with different lora, but had no luck. Could anyone answer this question? Thank you!

MaxLanc commented 1 year ago

You kinda "can", not like the extension in auto1111 tho, but by pure cope and hope using the nodes. TLDR you connect the model with all the loras to the KSampler but with like 0.5 strength on each lora and each region prompt has connected the model clip and their own lora clip as well.

You are not limiting the lora influence, just letting them interfere enough to have their intended effects and then on the high res pass you kinda can fix the artifacts and weirdness.

daizhuo commented 1 year ago

You kinda "can", not like the extension in auto1111 tho, but by pure cope and hope using the nodes. TLDR you connect the model with all the loras to the KSampler but with like 0.5 strength on each lora and each region prompt has connected the model clip and their own lora clip as well.

You are not limiting the lora influence, just letting them interfere enough to have their intended effects and then on the high res pass you kinda can fix the artifacts and weirdness.

Great 👍 Thank you so much! Could you attach one simple sample how to do that? That would be awesome!

MaxLanc commented 1 year ago

Sure thing, fair warning the results won't be as crisp and clear as they should but so far this is the only way without actually messing with the code lol

This is one such result ComfyUI_00064_

And this is the copium machine I noded in comfyUI, is pretty normal but it took me some seethe mald and cope to achieve lol imagen imagen imagen

MaxLanc commented 1 year ago

Great 👍 Thank you so much! Could you attach one simple sample how to do that? That would be awesome!

Forgot the quote, check answer up lol