Suggestion: Stable Diffusion Model

[jafri]

Been following along with your speed increases on Whisper using ggml, which have been amazing

Would be interesting to see how stable diffusion runs on CPUs using ggml

Here are current benchmarks:

Popular way to run SD: https://github.com/huggingface/diffusers

[ggerganov]

Yup, I've been thinking about it - the network seems a bit more complex compared to GPT and Whisper, but I think most of the building blocks are already available in ggml.

Last time I took a look at the python source, I got a bit lost because the model does not seem concentrated in one place as it is with the other models. For example, here I see 8 pipelines and I don't know which one is relevant and which not.

[jafri]

These 3 are the main pipelines

1: Text to Image (Most popular), give it a text prompt and it generates an image 2: Image to Image -> Example: give it an initial reference image and turn it into picasso style

3. Inpainting

[jafri]

This one is interesting too:

https://github.com/TheLastBen/fast-stable-diffusion

[x-legion]

Yup, I've been thinking about it - the network seems a bit more complex compared to GPT and Whisper, but I think most of the building blocks are already available in ggml.

Last time I took a look at the python source, I got a bit lost because the model does not seem concentrated in one place as it is with the other models. For example, here I see 8 pipelines and I don't know which one is relevant and which not.

[tikikun]

hi thumb up for this

[leejet]

I've implemented Stable Diffusion using GGML. Welcome to try it out.

https://github.com/leejet/stable-diffusion.cpp

This is an example

./sd -m ../models/sd-v1-4-ggml-model-f16.bin -p "a lovely cat"