Separate ControlNet / VAE / Lora / Diffusion model loading.

[LSXAxeller]

In the current implementation, models like ControlNet, VAE, Lora, Embeddings, and Tiny AutoEncoder (Taesd) can only be loaded alongside the Stable Diffusion model. This means that switching between different ControlNet models (e.g., Canny, Depth, OpenPose) or other model types requires completely reloading the Stable Diffusion model, which is inefficient, disruptive and time consuming.

Is there a way to allows these models to be loaded independently from the Stable Diffusion model? This would enable users to quickly swap between different model configurations without restarting the entire Stable Diffusion process, significantly improving workflow flexibility

[DarthAffe]

This is currently not possible. stable-diffusion.cpp does only offer a single entry point to create a "context". To my knowledge there's no way to replace/reload anything without creating a new context. (API-reference: https://github.com/leejet/stable-diffusion.cpp/blob/4a6e36edc586779918535e12b4fbe0583044ee6f/stable-diffusion.h#L121)

To support this it would first need to be changed in stable-diffusion.cpp. I guess this is kinda related: https://github.com/leejet/stable-diffusion.cpp/issues/252

[LSXAxeller]

This is currently not possible. stable-diffusion.cpp does only offer a single entry point to create a "context". To my knowledge there's no way to replace/reload anything without creating a new context. (API-reference: leejet/stable-diffusion.cpp@4a6e36e/stable-diffusion.h#L121)

To support this it would first need to be changed in stable-diffusion.cpp. I guess this is kinda related: leejet/stable-diffusion.cpp#252

thanks for your reply, since it's a stable-diffusion.cpp limitation, I am gonna close this issue and submit it at stable-diffusion.cpp repo, I hope that we get this feature soon