Separate ControlNet / VAE / Lora / Diffusion model loading.

[LSXAxeller]

In the current implementation (leejet/stable-diffusion.cpp@4a6e36e/stable-diffusion.h#L121), models like ControlNet, VAE, Lora, Embeddings, and Tiny AutoEncoder (Taesd) can only be loaded alongside the Stable Diffusion model. This means that switching between different ControlNet models (e.g., Canny, Depth, OpenPose) or other model types requires completely reloading the Stable Diffusion model, which is inefficient, disruptive and time consuming.

Is there a way to allows these models to be loaded independently from the Stable Diffusion model? This would enable users to quickly swap between different model configurations without restarting the entire Stable Diffusion process, significantly improving workflow flexibility

[Green-Sky]

This got implemented as part of flux support and now lives on master. I looks something like this: sd.exe --diffusion-model ..\models\flux1-dev-q8_0.gguf --vae ..\models\ae.sft --clip_l ..\models\clip_l.safetensors --t5xxl ..\models\t5xxl_fp16.safetensors ......

@leejet --clip_l, --t5xxl and --diffusion-model are missing from help. As well as other options...

[DarthAffe]

I might have missunderstood something, but to me this does not sound like it was the point of the question.

In the current master I still don't see a way to change the vae without reloading the diffusion model too (assuming you're directly using the lib, not the example-application), since I'd need to create a new context.

[Green-Sky]

Oh, my bad. Yea changing after creating is not supported right now. My brain just read "Separate [...] Diffusion model loading". :sweat_smile:

[LSXAxeller]

still no updates for this ? and DarthAffe is correct about this, I am using the lib through the C# bindings