search

leejet / stable-diffusion.cpp

Stable Diffusion and Flux in pure C/C++
MIT License
3.33k stars 280 forks source link

Image generation error on M1 and low image quanlity #222

Open Ucag opened 5 months ago

Ucag commented 5 months ago

I use SD_METAL=ON to build sd.cpp on m1. However, the generated image seems to be blured or tends to be in the cartoon-like style. This happens on all models (v2, SDXL, SDXL turbo). This problem only shows on m1. When change to other platforms, the image works fine. For making sure the model weight works, I made a comparison using sd.cpp on m1, sd.cpp on ubuntu nvidia gpu, pytorch model on nvidia gpu with same prompt "a lovely cat" and default seed.

bad_images

This problem sees worse when using SDXL series model ( SDXL base, SDXL turbo, etc) I noticed that I need to apply the fixed vae model. However, things not worked in an expected way.

sdxl_turbo_with_vae

I've tried to fix the problem. I guess there may have something to do with the ggml itself. The very first thing to do is to upgrade ggml version of sdcpp. However, it seems that the quantize/dequantize policy of scheduling different operators changed. The logs shows as below:

image

I failed to upgrade it to latest ggml version.

What's more, it takes 4~5 seconds to run one iteration on m1, is this normal or expected way?

Green-Sky commented 5 months ago

looks like you are not setting --cfg-scale appropriately, at least for turbo.

Ucag commented 5 months ago

looks like you are not setting --cfg-scale appropriately, at least for turbo.

Actually I've tried other --cfg-scale values, they go to the cartoon-like reuslt. Like below:

image