search

HighCWu / control-lora-v2

ControlLoRA Version 2: A Lightweight Neural Network To Control Stable Diffusion Spatial Information Version 2
Apache License 2.0
98 stars 6 forks source link

Training control Lora for SDXL #3

Open celticsha opened 7 months ago

celticsha commented 7 months ago

How do I use this for SDXL please?

HighCWu commented 7 months ago

This code currently only works with stable diffusion v1.x. If you are interested in training it under sdxl, you can try to implement it. The principle is very simple. You only need to construct the controlnet of sdxl from the weights of sdxl, freeze its weights, and bind lora to the linear layer and convolutional layer of the controlnet.

celticsha commented 7 months ago

Thank you. I am a user rather than a coder, so it's not as easy for me to implement that as you may assume. I'll perhaps give it a go.