search

HighCWu / control-lora-v3

ControlLoRA Version 3: LoRA Is All You Need to Control the Spatial Information of Stable Diffusion.
MIT License
27 stars 0 forks source link

multiple control-loras with pretrained stabilityai/control-lora #4

Open smuelpeng opened 19 hours ago

smuelpeng commented 19 hours ago

Thank you very much for your excellent work; I am deeply inspired. However, I am really interested in knowing whether your method is compatible with the following models:

control-lora-canny-rank256.safetensors control-lora-depth-rank256.safetensors control-lora-openposeXL2-rank256.safetensors These control-lora models can be easily used in ComfyUI, but I haven’t been able to find a viable method for achieving the same in code. I primarily use the StableDiffusionXLControlNetImg2ImgPipeline from Diffusers to optimize images using multiple controlnets, aiming to replicate the functionality of using multiple control-loras in ComfyUI.

I would greatly appreciate it if you could provide guidance on how to load and use these existing control-loras. If you could offer demo code or pseudo-code, I would be extremely grateful, as this issue has been troubling me for a long time.

looking forward to your response！

HighCWu commented 19 hours ago

I'm very sorry to tell you that their method is not compatible with mine. My control-lora-v2 is more similar to their method, but not the same. So you can't use my method to load or train their model