cb88
commented
2 months ago Not sure this is the issue you are running into but for instance flux apparently needs the shift value set otherwise it gets blurry... which it appears S-D.cpp does not have a parameter for currently. It might make sense to have this as a both a parameter to set custom shift as well as a flag to auto set shift for flux models that need it, if you have both turned the custom parameter could increase or decrease the auto calculated shift value.
function calcShift(h, w) { const step1 = (h w) / 256; const step2 = (1.15 - 0.5) / (4096 - 256); const step3 = (step1 - 256) step2 + 0.5; const result = Math.exp(step3); return Math.round(result * 100) / 100; }
https://www.reddit.com/r/drawthingsapp/comments/1erjvur/flux1_dev_8bit_generation_is_blurry/
Example blurry output at 30 steps with q8 flux dev:
Hello :) I am using compiled sd.cpp with SYCL kernel since i have intel arc gpu
so i started with conversion model from https://civitai.com/models/141592/pixelwave (which is 22+ gb) and it finished without any errors
log of conversion process
```python Option: n_threads: 8 mode: convert model_path: /home/models_test/pixelwave_flux1Dev02.safetensors wtype: q8_0 clip_l_path: t5xxl_path: diffusion_model_path: vae_path: taesd_path: esrgan_path: controlnet_path: embeddings_path: stacked_id_embeddings_path: input_id_images_path: style ratio: 20.00 normalize input image : false output_path: /home/unet/pixelwave_flux1Dev02_Q8_0.gguf init_img: control_image: clip on cpu: false controlnet cpu: false vae decoder on cpu:false strength(control): 0.90 prompt: negative_prompt: min_cfg: 1.00 cfg_scale: 7.00 guidance: 3.50 clip_skip: -1 width: 512 height: 512 sample_method: euler_a schedule: default sample_steps: 20 strength(img2img): 0.75 rng: cuda seed: 42 batch_count: 1 vae_tiling: false upscale_repeats: 1 System Info: BLAS = 1 SSE3 = 1 AVX = 1 AVX2 = 1 AVX512 = 0 AVX512_VBMI = 0 AVX512_VNNI = 0 FMA = 1 NEON = 0 ARM_FMA = 0 F16C = 1 FP16_VA = 0 WASM_SIMD = 0 VSX = 0 [INFO ] model.cpp:793 - load /home/models_test/pixelwave_flux1Dev02.safetensors using safetensors format [DEBUG] model.cpp:861 - init from ' /home/models_test/pixelwave_flux1Dev02.safetensors' [INFO ] model.cpp:1776 - model tensors mem size: 12248.99MB [DEBUG] model.cpp:1530 - loading tensors from /home/models_test/pixelwave_flux1Dev02.safetensors [INFO ] model.cpp:1811 - load tensors done [INFO ] model.cpp:1812 - trying to save tensors to /home/unet/pixelwave_flux1Dev02_Q8_0.gguf convert ' /home/models_test/pixelwave_flux1Dev02.safetensors'/'' to ' /home/unet/pixelwave_flux1Dev02_Q8_0.gguf' success ```then i am trying to generate image with sd.cpp using this command and no errors again so far while generating image
log of image generation process
```python Option: n_threads: 8 mode: txt2img model_path: wtype: unspecified clip_l_path: /home/clip/clip_l.safetensors t5xxl_path: /home/clip/clip_t5xxl_fp16.safetensors diffusion_model_path: /home/unet/pixelwave_flux1Dev02_Q8_0.gguf vae_path: /home/vae/flux_vae.safetensors taesd_path: esrgan_path: controlnet_path: embeddings_path: stacked_id_embeddings_path: input_id_images_path: style ratio: 20.00 normalize input image : false output_path: /tmp/output.png init_img: control_image: clip on cpu: false controlnet cpu: false vae decoder on cpu:true strength(control): 0.90 prompt: The transparent orb creates an intriguing, otherworldly atmosphere and allows viewers to peer into the fantasy world within negative_prompt: min_cfg: 1.00 cfg_scale: 1.00 guidance: 3.50 clip_skip: -1 width: 1024 height: 1024 sample_method: euler schedule: discrete sample_steps: 16 strength(img2img): 0.75 rng: cuda seed: 42 batch_count: 1 vae_tiling: false upscale_repeats: 1 System Info: BLAS = 1 SSE3 = 1 AVX = 1 AVX2 = 1 AVX512 = 0 AVX512_VBMI = 0 AVX512_VNNI = 0 FMA = 1 NEON = 0 ARM_FMA = 0 F16C = 1 FP16_VA = 0 WASM_SIMD = 0 VSX = 0 [DEBUG] stable-diffusion.cpp:175 - Using SYCL backend [SYCL] call ggml_check_sycl ggml_check_sycl: GGML_SYCL_DEBUG: 0 ggml_check_sycl: GGML_SYCL_F16: no ZE_LOADER_DEBUG_TRACE:Using Loader Library Path: ZE_LOADER_DEBUG_TRACE:Tracing Layer Library Path: libze_tracing_layer.so.1 found 1 SYCL devices: | | | | |Max | |Max |Global | | | | | | |compute|Max work|sub |mem | | |ID| Device Type| Name|Version|units |group |group|size | Driver version| |--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------| | 0| [level_zero:gpu:0]| Intel Arc A770 Graphics| 1.5| 512| 1024| 32| 16225M| 1.3.30872| ggml_sycl_init: GGML_SYCL_FORCE_MMQ: no ggml_sycl_init: SYCL_USE_XMX: yes ggml_sycl_init: found 1 SYCL devices: [WARN ] stable-diffusion.cpp:185 - Flash Attention not supported with GPU Backend [INFO ] stable-diffusion.cpp:202 - loading clip_l from ' /home/clip/clip_l.safetensors' [INFO ] model.cpp:793 - load /home/clip/clip_l.safetensors using safetensors format [DEBUG] model.cpp:861 - init from ' /home/clip/clip_l.safetensors' [INFO ] stable-diffusion.cpp:209 - loading t5xxl from ' /home/clip/clip_t5xxl_fp16.safetensors' [INFO ] model.cpp:793 - load /home/clip/clip_t5xxl_fp16.safetensors using safetensors format [DEBUG] model.cpp:861 - init from ' /home/clip/clip_t5xxl_fp16.safetensors' [INFO ] stable-diffusion.cpp:216 - loading diffusion model from ' /home/unet/pixelwave_flux1Dev02_Q8_0.gguf' [INFO ] model.cpp:790 - load /home/unet/pixelwave_flux1Dev02_Q8_0.gguf using gguf format [DEBUG] model.cpp:807 - init from ' /home/unet/pixelwave_flux1Dev02_Q8_0.gguf' WARNING: Behavior may be unexpected when allocating 0 bytes for ggml_calloc! [INFO ] stable-diffusion.cpp:223 - loading vae from ' /home/vae/flux_vae.safetensors' [INFO ] model.cpp:793 - load /home/vae/flux_vae.safetensors using safetensors format [DEBUG] model.cpp:861 - init from ' /home/vae/flux_vae.safetensors' [INFO ] stable-diffusion.cpp:235 - Version: Flux Dev [INFO ] stable-diffusion.cpp:266 - Weight type: f16 [INFO ] stable-diffusion.cpp:267 - Conditioner weight type: f16 [INFO ] stable-diffusion.cpp:268 - Diffusion model weight type: q8_0 [INFO ] stable-diffusion.cpp:269 - VAE weight type: f32 [DEBUG] stable-diffusion.cpp:271 - ggml tensor size = 400 bytes [INFO ] stable-diffusion.cpp:310 - set clip_on_cpu to true [INFO ] stable-diffusion.cpp:313 - CLIP: Using CPU backend [DEBUG] clip.hpp:171 - vocab size: 49408 [DEBUG] clip.hpp:182 - trigger word img already in vocab [DEBUG] ggml_extend.hpp:1050 - clip params backend buffer size = 235.06 MB(RAM) (196 tensors) [DEBUG] ggml_extend.hpp:1050 - t5 params backend buffer size = 9083.77 MB(RAM) (219 tensors) [DEBUG] ggml_extend.hpp:1050 - flux params backend buffer size = 12068.09 MB(VRAM) (780 tensors) [INFO ] stable-diffusion.cpp:334 - VAE Autoencoder: Using CPU backend [DEBUG] ggml_extend.hpp:1050 - vae params backend buffer size = 94.57 MB(RAM) (138 tensors) [DEBUG] stable-diffusion.cpp:398 - loading weights [DEBUG] model.cpp:1530 - loading tensors from /home/clip/clip_l.safetensors [DEBUG] model.cpp:1530 - loading tensors from /home/clip/clip_t5xxl_fp16.safetensors [INFO ] model.cpp:1685 - unknown tensor 'text_encoders.t5xxl.encoder.embed_tokens.weight | f16 | 2 [4096, 32128, 1, 1, 1]' in model file [DEBUG] model.cpp:1530 - loading tensors from /home/unet/pixelwave_flux1Dev02_Q8_0.gguf [DEBUG] model.cpp:1530 - loading tensors from /home/vae/flux_vae.safetensors [INFO ] stable-diffusion.cpp:497 - total params memory size = 21481.50MB (VRAM 12068.09MB, RAM 9413.41MB): clip 9318.83MB(RAM), unet 12068.09MB(VRAM), vae 94.57MB(RAM), controlnet 0.00MB(VRAM), pmid 0.00MB(RAM) [INFO ] stable-diffusion.cpp:501 - loading model from '' completed, taking 6.08s [INFO ] stable-diffusion.cpp:518 - running in Flux FLOW mode [INFO ] stable-diffusion.cpp:534 - running with discrete schedule [DEBUG] stable-diffusion.cpp:572 - finished loaded file [DEBUG] stable-diffusion.cpp:1378 - txt2img 1024x1024 [DEBUG] stable-diffusion.cpp:1127 - prompt after extract and remove lora: "The transparent orb creates an intriguing, otherworldly atmosphere and allows viewers to peer into the fantasy world within" [INFO ] stable-diffusion.cpp:655 - Attempting to apply 0 LoRAs [INFO ] stable-diffusion.cpp:1132 - apply_loras completed, taking 0.00s [DEBUG] conditioner.hpp:1036 - parse 'The transparent orb creates an intriguing, otherworldly atmosphere and allows viewers to peer into the fantasy world within' to [['The transparent orb creates an intriguing, otherworldly atmosphere and allows viewers to peer into the fantasy world within', 1], ] [DEBUG] clip.hpp:311 - token length: 77 [DEBUG] t5.hpp:397 - token length: 256 [DEBUG] ggml_extend.hpp:1001 - t5 compute buffer size: 68.25 MB(RAM) [DEBUG] conditioner.hpp:1155 - computing condition graph completed, taking 6029 ms [INFO ] stable-diffusion.cpp:1256 - get_learned_condition completed, taking 6030 ms [INFO ] stable-diffusion.cpp:1279 - sampling using Euler method [INFO ] stable-diffusion.cpp:1283 - generating image: 1/1 - seed 42 [DEBUG] ggml_extend.hpp:1001 - flux compute buffer size: 2577.25 MB(VRAM) |===> | 1/16 - 6.89s/it |======> | 2/16 - 6.44s/it |=========> | 3/16 - 6.48s/it |============> | 4/16 - 6.55s/it |===============> | 5/16 - 6.48s/it |==================> | 6/16 - 6.43s/it |=====================> | 7/16 - 6.41s/it |=========================> | 8/16 - 6.44s/it |============================> | 9/16 - 6.49s/it |===============================> | 10/16 - 6.53s/it |==================================> | 11/16 - 6.52s/it |=====================================> | 12/16 - 6.52s/it |========================================> | 13/16 - 6.58s/it |===========================================> | 14/16 - 6.45s/it |==============================================> | 15/16 - 6.48s/it |==================================================| 16/16 - 6.42s/it [INFO ] stable-diffusion.cpp:1315 - sampling completed, taking 104.18s [INFO ] stable-diffusion.cpp:1323 - generating 1 latent images completed, taking 104.20s [INFO ] stable-diffusion.cpp:1326 - decoding 1 latents [DEBUG] ggml_extend.hpp:1001 - vae compute buffer size: 6656.00 MB(RAM) [DEBUG] stable-diffusion.cpp:987 - computing vae [mode: DECODE] graph completed, taking 46.52s [INFO ] stable-diffusion.cpp:1336 - latent 1 decoded, taking 46.52s [INFO ] stable-diffusion.cpp:1340 - decode_first_stage completed, taking 46.52s [INFO ] stable-diffusion.cpp:1449 - txt2img completed in 156.75s save result image to '/tmp/output.png' ```and in the end i am getting this kind of gibberish image
result image
How to fix this image generation process? Or i need to change something in conversion? If i trying to generate with ComfyUI the result is same.