search

EdVince / Stable-Diffusion-NCNN

Stable Diffusion in NCNN with c++, supported txt2img and img2img
BSD 3-Clause "New" or "Revised" License
952 stars 94 forks source link

@fenwang's 7 GB to 5.5 GB (512x512) #25

Open ClashSAN opened 1 year ago

ClashSAN commented 1 year ago

https://github.com/fengwang/Stable-Diffusion-NCNN image hi, It looked like there was this memory optimization applicable to 8gb laptops (no swap requirement)

Is this also applicable to android mobile, specifically? You have mentioned a while ago the f16 512x512 version working on your 12gb ram android. Also, is the 256x256 model a fixed shape, and is a quantized model?

EdVince commented 1 year ago

For some of the metrices, I have not updated them for some time and suggest you actually test them. All the models I have used are fp16, not quantitative one.

ClashSAN commented 1 year ago

thanks. my bad, the models were obviously labeled. I used your diffusers conversion repository successfully, may I ask if the current vae decoder provided via cloud drive is the separate nai vae, or the one built-in (regular SD)? The int8 and/or uint8 quantization process is easy enough with onnx, but I don't know how to do this with .pt (pnnx)

Your ncnn implementation is currently as fast as the openvino implementation, and supports multiple sizes. I am interested in quantization because you have an apk release that supports 6gb currently, and 4gb may be supported if the quantized model is used, or maybe some low ram optimization. I'm not sure.

EdVince commented 1 year ago

But considering that diffusion is a noisy computational process, I don't think quantifying such data will give a good result.