search

leejet / stable-diffusion.cpp

Stable Diffusion and Flux in pure C/C++
MIT License
3.25k stars 273 forks source link

Feature Requests #342

Open KintCark opened 1 month ago

KintCark commented 1 month ago

Can u add the ability to save the qauntized models to storage that way if we want to use it again we don't have to keep reconverting every time we start an generation.

grauho commented 1 month ago

You should already be able to do that by setting the "-M, --mode" argument to "convert" to save the model as a quantized gguf file at the location specified with "-o, --output".

More information in the docs: docs/quantization_and_gguf.md

KintCark commented 4 weeks ago

It auto saves to the main folder but I tried qauntized aura flow buy it killed when saving output

grauho commented 4 weeks ago

Interesting, please recompile with --config Debug, re-run with --verbose, and post the output

KintCark commented 3 weeks ago

Interesting, please recompile with --config Debug, re-run with --verbose, and post the output

I can't post output termux crashes immediately I got 7gb ram to spare but it loads the tensors but as soon as it trys to save output it crashes. Could someone else try converting aura flow 0.3 please I need q5 or q4 and q8

grauho commented 3 weeks ago

I'm not familiar with aura flow but I wonder if it's based on a model type that sdcpp doesn't currently support. You could pipe the output to a file to avoid losing it when termux crashes, eg: "./sd etc etc &> foo.txt" to pipe both stdout and stderr to a file.

KintCark commented 3 weeks ago

I'm not familiar with aura flow but I wonder if it's based on a model type that sdcpp doesn't currently support. You could pipe the output to a file to avoid losing it when termux crashes, eg: "./sd etc etc &> foo.txt" to pipe both stdout and stderr to a file.

You're right that has to be what it is it's not supported so it don't work so I can quantize sd3 I bet. Will flux and aura flow be added soon?

KintCark commented 3 weeks ago

Gguf used less memory I can run flux q8_0 and t5xxlfp16 in comfyui on my phone so using gguf is better than safetensors

KintCark commented 3 weeks ago

Sd3 uses flow so how come flux and aura not auto support?

grauho commented 3 weeks ago

Sd3 uses flow so how come flux and aura not auto support?

From my understanding of the similarities between Flux and SD3 is that while they share some components, they are not identical in their architecture. So, that's why despite SD3 being available in sdcpp there is no "auto support" for Flux.