search

leejet / stable-diffusion.cpp

Stable Diffusion and Flux in pure C/C++
MIT License
3.25k stars 273 forks source link

Flux do not output anything #385

Open stereomatchingkiss opened 2 weeks ago

stereomatchingkiss commented 2 weeks ago

Using win vulkan, master-f4c937c

Download the weights following flux.md

gpu : 3070Ti

Command

./sd.exe --diffusion-model ./flux/schnell/flux1-schnell-q2_k.gguf --vae ./flux/ae.safetensors --clip_l ./flux/clip_l.safetensors --t5xxl ./flux/t5xxl_fp16.safetensors -p "a lovely cat holding a sign says 'flux.cpp'" --cfg-scale 1.0 --sampling-method euler -v --steps 4

Output message

Option: n_threads: 6 mode: txt2img model_path: wtype: unspecified clip_l_path: ./flux/clip_l.safetensors t5xxl_path: ./flux/t5xxl_fp16.safetensors diffusion_model_path: ./flux/schnell/flux1-schnell-q2_k.gguf vae_path: ./flux/ae.safetensors taesd_path: esrgan_path: controlnet_path: embeddings_path: stacked_id_embeddings_path: input_id_images_path: style ratio: 20.00 normalize input image : false output_path: output.png init_img: control_image: clip on cpu: false controlnet cpu: false vae decoder on cpu:false strength(control): 0.90 prompt: a lovely cat holding a sign says 'flux.cpp' negative_prompt: min_cfg: 1.00 cfg_scale: 1.00 guidance: 3.50 clip_skip: -1 width: 512 height: 512 sample_method: euler schedule: default sample_steps: 4 strength(img2img): 0.75 rng: cuda seed: 42 batch_count: 1 vae_tiling: false upscale_repeats: 1 System Info: BLAS = 1 SSE3 = 1 AVX = 1 AVX2 = 1 AVX512 = 0 AVX512_VBMI = 0 AVX512_VNNI = 0 FMA = 1 NEON = 0 ARM_FMA = 0 F16C = 1 FP16_VA = 0 WASM_SIMD = 0 VSX = 0 [DEBUG] stable-diffusion.cpp:166 - Using Vulkan backend ggml_vulkan: Found 1 Vulkan devices: Vulkan0: NVIDIA GeForce RTX 3070 Ti (NVIDIA) | uma: 0 | fp16: 1 | warp size: 32 [INFO ] stable-diffusion.cpp:202 - loading clip_l from './flux/clip_l.safetensors' [INFO ] model.cpp:793 - load ./flux/clip_l.safetensors using safetensors format [DEBUG] model.cpp:861 - init from './flux/clip_l.safetensors' [INFO ] stable-diffusion.cpp:209 - loading t5xxl from './flux/t5xxl_fp16.safetensors' [INFO ] model.cpp:793 - load ./flux/t5xxl_fp16.safetensors using safetensors format [DEBUG] model.cpp:861 - init from './flux/t5xxl_fp16.safetensors' [INFO ] stable-diffusion.cpp:216 - loading diffusion model from './flux/schnell/flux1-schnell-q2_k.gguf' [INFO ] model.cpp:790 - load ./flux/schnell/flux1-schnell-q2_k.gguf using gguf format [DEBUG] model.cpp:807 - init from './flux/schnell/flux1-schnell-q2_k.gguf' WARNING: Behavior may be unexpected when allocating 0 bytes for ggml_calloc! [INFO ] stable-diffusion.cpp:223 - loading vae from './flux/ae.safetensors' [INFO ] model.cpp:793 - load ./flux/ae.safetensors using safetensors format [DEBUG] model.cpp:861 - init from './flux/ae.safetensors' [INFO ] stable-diffusion.cpp:235 - Version: Flux Schnell [INFO ] stable-diffusion.cpp:266 - Weight type: f16 [INFO ] stable-diffusion.cpp:267 - Conditioner weight type: f16 [INFO ] stable-diffusion.cpp:268 - Diffusion model weight type: q2_K [INFO ] stable-diffusion.cpp:269 - VAE weight type: f32 [DEBUG] stable-diffusion.cpp:271 - ggml tensor size = 400 bytes [INFO ] stable-diffusion.cpp:310 - set clip_on_cpu to true [INFO ] stable-diffusion.cpp:313 - CLIP: Using CPU backend [DEBUG] clip.hpp:171 - vocab size: 49408 [DEBUG] clip.hpp:182 - trigger word img already in vocab [DEBUG] ggml_extend.hpp:1050 - clip params backend buffer size = 235.06 MB(RAM) (196 tensors) [DEBUG] ggml_extend.hpp:1050 - t5 params backend buffer size = 9083.77 MB(RAM) (219 tensors) [DEBUG] ggml_extend.hpp:1050 - flux params backend buffer size = 3732.51 MB(VRAM) (776 tensors) [DEBUG] ggml_extend.hpp:1050 - vae params backend buffer size = 94.57 MB(VRAM) (138 tensors) [DEBUG] stable-diffusion.cpp:398 - loading weights [DEBUG] model.cpp:1530 - loading tensors from ./flux/clip_l.safetensors [DEBUG] model.cpp:1530 - loading tensors from ./flux/t5xxl_fp16.safetensors [INFO ] model.cpp:1685 - unknown tensor 'text_encoders.t5xxl.encoder.embed_tokens.weight | f16 | 2 [4096, 32128, 1, 1, 1]' in model file [DEBUG] model.cpp:1530 - loading tensors from ./flux/schnell/flux1-schnell-q2_k.gguf [DEBUG] model.cpp:1530 - loading tensors from ./flux/ae.safetensors [INFO ] stable-diffusion.cpp:497 - total params memory size = 13145.92MB (VRAM 3827.08MB, RAM 9318.83MB): clip 9318.83MB(RAM), unet 3732.51MB(VRAM), vae 94.57MB(VRAM), controlnet 0.00MB(VRAM), pmid 0.00MB(RAM) [INFO ] stable-diffusion.cpp:501 - loading model from '' completed, taking 184.50s [INFO ] stable-diffusion.cpp:518 - running in Flux FLOW mode [DEBUG] stable-diffusion.cpp:572 - finished loaded file [DEBUG] stable-diffusion.cpp:1378 - txt2img 512x512 [DEBUG] stable-diffusion.cpp:1127 - prompt after extract and remove lora: "a lovely cat holding a sign says 'flux.cpp'" [INFO ] stable-diffusion.cpp:655 - Attempting to apply 0 LoRAs [INFO ] stable-diffusion.cpp:1132 - apply_loras completed, taking 0.00s [DEBUG] conditioner.hpp:1036 - parse 'a lovely cat holding a sign says 'flux.cpp'' to [['a lovely cat holding a sign says 'flux.cpp'', 1], ] [DEBUG] clip.hpp:311 - token length: 77 [DEBUG] t5.hpp:397 - token length: 256 [DEBUG] ggml_extend.hpp:1001 - t5 compute buffer size: 68.25 MB(RAM) [DEBUG] conditioner.hpp:1155 - computing condition graph completed, taking 15083 ms [INFO ] stable-diffusion.cpp:1256 - get_learned_condition completed, taking 15093 ms [INFO ] stable-diffusion.cpp:1279 - sampling using Euler method [INFO ] stable-diffusion.cpp:1283 - generating image: 1/1 - seed 42 [DEBUG] ggml_extend.hpp:1001 - flux compute buffer size: 398.50 MB(VRAM)

Looks like it do not start the image generation process, not sure what is happening

Following command work

./sd.exe -m sd-v1-4.ckpt --cfg-scale 5 --steps 30 --sampling-method euler -H 512 -W 512 --seed 42 -p "fantasy medieval village world inside a glass sphere , high detail, fantasy, realistic, light effect, hyper detail, volumetric lighting, cinematic, macro, depth of field, blur, red light and clouds from the back, highly detailed epic cinematic concept art cg render made in maya, blender and photoshop, octane render, excellent composition, dynamic dramatic cinematic lighting, aesthetic, very inspirational, world inside a glass sphere by james gurney by artgerm with james jean, joe fenton and tristan eaton by ross tran, fine details, 4k resolution"

Green-Sky commented 2 weeks ago

@stereomatchingkiss yea the log looks correct, after a while, it should start to print

[INFO ] stable-diffusion.cpp:1279 - sampling using Euler method
[INFO ] stable-diffusion.cpp:1283 - generating image: 1/6 - seed 1297707041
  |==================================================| 8/8 - 6.81s/it
[INFO ] stable-diffusion.cpp:1315 - sampling completed, taking 54.60s

or similar.

also please dont click on any of those spam links.

@leejet can you give me some moderation power so i can remove those malicious links when they pop up?

stereomatchingkiss commented 2 weeks ago

@stereomatchingkiss yea the log looks correct, after a while, it should start to print

[INFO ] stable-diffusion.cpp:1279 - sampling using Euler method
[INFO ] stable-diffusion.cpp:1283 - generating image: 1/6 - seed 1297707041
  |==================================================| 8/8 - 6.81s/it
[INFO ] stable-diffusion.cpp:1315 - sampling completed, taking 54.60s

or similar.

also please dont click on any of those spam links.

@leejet can you give me some moderation power so i can remove those malicious links when they pop up?

Thanks and worry not, I do not click on them. They looks suspicious for me.

By the way, after a while the program end, nothing output. I change the clip to "t5xxl_fp8_e4m3fn.safetensors", but still the same. ComFyUI works well on my pc, using same model.

Green-Sky commented 2 weeks ago

hm wierd. can you download preconverted ggufs and see if that makes a difference? Also, if the model files are bf16 or fp8_e4m3 they get upconverted inplace (bf16->fp32, e4m3->f16), so i recommend converting them beforehand.

stduhpf commented 2 weeks ago

I think Flux won't work with Vulkan until https://github.com/leejet/stable-diffusion.cpp/pull/378 gets merged

stereomatchingkiss commented 1 week ago

hm wierd. can you download preconverted ggufs and see if that makes a difference?

Tried, same results. Will try again after #378 merged.

SkutteOleg commented 1 week ago

You can try https://github.com/SkutteOleg/stable-diffusion.cpp/releases/tag/master-cff6645 where it's already merged

stereomatchingkiss commented 1 week ago

You can try https://github.com/SkutteOleg/stable-diffusion.cpp/releases/tag/master-cff6645 where it's already merged

Tried with

./sd.exe --diffusion-model ./flux/schnell/flux1-schnell-q2_k.gguf --vae ./flux/ae.safetensors --clip_l ./flux/clip_l.safetensors --t5xxl D./flux/t5xxl_fp8_e4m3fn.safetensors -p "a lovely cat holding a sign says 'flux.cpp'" --cfg-scale 1.0 --sampling-method euler -v --steps 4

Output

Option: n_threads: 6 mode: txt2img model_path: wtype: unspecified clip_l_path: ./flux/clip_l.safetensors t5xxl_path: D./flux/t5xxl_fp8_e4m3fn.safetensors diffusion_model_path: ./flux/schnell/flux1-schnell-q2_k.gguf vae_path: ./flux/ae.safetensors taesd_path: esrgan_path: controlnet_path: embeddings_path: stacked_id_embeddings_path: input_id_images_path: style ratio: 20.00 normalize input image : false output_path: output.png init_img: control_image: clip on cpu: false controlnet cpu: false vae decoder on cpu:false strength(control): 0.90 prompt: a lovely cat holding a sign says 'flux.cpp' negative_prompt: min_cfg: 1.00 cfg_scale: 1.00 guidance: 3.50 clip_skip: -1 width: 512 height: 512 sample_method: euler schedule: default sample_steps: 4 strength(img2img): 0.75 rng: cuda seed: 42 batch_count: 1 vae_tiling: false upscale_repeats: 1 System Info: BLAS = 1 SSE3 = 1 AVX = 1 AVX2 = 1 AVX512 = 0 AVX512_VBMI = 0 AVX512_VNNI = 0 FMA = 1 NEON = 0 ARM_FMA = 0 F16C = 1 FP16_VA = 0 WASM_SIMD = 0 VSX = 0 [DEBUG] stable-diffusion.cpp:166 - Using Vulkan backend ggml_vulkan: Found 1 Vulkan devices: Vulkan0: NVIDIA GeForce RTX 3070 Ti (NVIDIA) | uma: 0 | fp16: 1 | warp size: 32 [INFO ] stable-diffusion.cpp:202 - loading clip_l from './flux/clip_l.safetensors' [INFO ] model.cpp:793 - load ./flux/clip_l.safetensors using safetensors format [DEBUG] model.cpp:861 - init from './flux/clip_l.safetensors' [INFO ] stable-diffusion.cpp:209 - loading t5xxl from 'D./flux/t5xxl_fp8_e4m3fn.safetensors' [WARN ] model.cpp:799 - unknown format D./flux/t5xxl_fp8_e4m3fn.safetensors [WARN ] stable-diffusion.cpp:211 - loading t5xxl from 'D./flux/t5xxl_fp8_e4m3fn.safetensors' failed [INFO ] stable-diffusion.cpp:216 - loading diffusion model from './flux/schnell/flux1-schnell-q2_k.gguf' [INFO ] model.cpp:790 - load ./flux/schnell/flux1-schnell-q2_k.gguf using gguf format [DEBUG] model.cpp:807 - init from './flux/schnell/flux1-schnell-q2_k.gguf' WARNING: Behavior may be unexpected when allocating 0 bytes for ggml_calloc! [INFO ] stable-diffusion.cpp:223 - loading vae from './flux/ae.safetensors' [INFO ] model.cpp:793 - load ./flux/ae.safetensors using safetensors format [DEBUG] model.cpp:861 - init from './flux/ae.safetensors' [INFO ] stable-diffusion.cpp:235 - Version: Flux Schnell [INFO ] stable-diffusion.cpp:266 - Weight type: f16 [INFO ] stable-diffusion.cpp:267 - Conditioner weight type: f16 [INFO ] stable-diffusion.cpp:268 - Diffusion model weight type: q2_K [INFO ] stable-diffusion.cpp:269 - VAE weight type: f32 [DEBUG] stable-diffusion.cpp:271 - ggml tensor size = 400 bytes [INFO ] stable-diffusion.cpp:310 - set clip_on_cpu to true [INFO ] stable-diffusion.cpp:313 - CLIP: Using CPU backend [DEBUG] clip.hpp:171 - vocab size: 49408 [DEBUG] clip.hpp:182 - trigger word img already in vocab [DEBUG] ggml_extend.hpp:1050 - clip params backend buffer size = 235.06 MB(RAM) (196 tensors) [DEBUG] ggml_extend.hpp:1050 - t5 params backend buffer size = 9083.77 MB(RAM) (219 tensors) [DEBUG] ggml_extend.hpp:1050 - flux params backend buffer size = 3732.51 MB(VRAM) (776 tensors) [DEBUG] ggml_extend.hpp:1050 - vae params backend buffer size = 94.57 MB(VRAM) (138 tensors) [DEBUG] stable-diffusion.cpp:398 - loading weights [DEBUG] model.cpp:1530 - loading tensors from ./flux/clip_l.safetensors [DEBUG] model.cpp:1530 - loading tensors from ./flux/schnell/flux1-schnell-q2_k.gguf [DEBUG] model.cpp:1530 - loading tensors from ./flux/ae.safetensors [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.0.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.0.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.0.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.0.layer.0.SelfAttention.relative_attention_bias.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.0.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.0.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.0.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.0.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.0.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.0.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.1.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.1.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.1.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.1.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.1.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.1.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.1.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.1.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.1.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.10.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.10.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.10.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.10.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.10.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.10.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.10.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.10.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.10.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.11.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.11.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.11.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.11.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.11.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.11.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.11.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.11.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.11.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.12.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.12.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.12.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.12.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.12.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.12.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.12.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.12.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.12.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.13.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.13.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.13.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.13.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.13.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.13.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.13.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.13.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.13.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.14.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.14.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.14.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.14.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.14.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.14.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.14.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.14.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.14.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.15.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.15.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.15.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.15.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.15.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.15.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.15.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.15.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.15.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.16.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.16.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.16.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.16.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.16.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.16.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.16.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.16.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.16.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.17.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.17.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.17.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.17.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.17.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.17.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.17.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.17.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.17.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.18.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.18.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.18.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.18.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.18.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.18.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.18.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.18.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.18.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.19.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.19.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.19.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.19.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.19.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.19.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.19.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.19.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.19.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.2.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.2.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.2.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.2.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.2.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.2.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.2.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.2.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.2.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.20.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.20.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.20.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.20.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.20.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.20.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.20.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.20.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.20.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.21.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.21.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.21.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.21.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.21.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.21.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.21.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.21.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.21.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.22.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.22.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.22.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.22.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.22.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.22.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.22.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.22.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.22.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.23.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.23.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.23.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.23.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.23.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.23.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.23.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.23.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.23.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.3.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.3.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.3.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.3.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.3.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.3.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.3.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.3.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.3.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.4.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.4.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.4.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.4.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.4.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.4.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.4.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.4.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.4.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.5.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.5.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.5.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.5.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.5.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.5.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.5.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.5.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.5.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.6.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.6.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.6.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.6.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.6.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.6.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.6.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.6.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.6.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.7.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.7.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.7.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.7.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.7.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.7.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.7.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.7.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.7.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.8.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.8.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.8.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.8.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.8.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.8.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.8.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.8.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.8.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.9.layer.0.SelfAttention.k.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.9.layer.0.SelfAttention.o.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.9.layer.0.SelfAttention.q.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.9.layer.0.SelfAttention.v.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.9.layer.0.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.9.layer.1.DenseReluDense.wi_0.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.9.layer.1.DenseReluDense.wi_1.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.9.layer.1.DenseReluDense.wo.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.block.9.layer.1.layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.encoder.final_layer_norm.weight' not in model file [ERROR] model.cpp:1729 - tensor 'text_encoders.t5xxl.shared.weight' not in model file [ERROR] stable-diffusion.cpp:420 - load tensors from model loader failed new_sd_ctx_t failed

I think I should not use q2 directly(some models may need to convert), so I tried with flux1-schnell-q8_0.gguf, and have this output

Option: n_threads: 6 mode: txt2img model_path: wtype: unspecified clip_l_path: ./flux/clip_l.safetensors t5xxl_path: D./flux/t5xxl_fp8_e4m3fn.safetensors diffusion_model_path: ./flux/schnell/flux1-schnell-q8_0.gguf vae_path: ./flux/ae.safetensors taesd_path: esrgan_path: controlnet_path: embeddings_path: stacked_id_embeddings_path: input_id_images_path: style ratio: 20.00 normalize input image : false output_path: output.png init_img: control_image: clip on cpu: false controlnet cpu: false vae decoder on cpu:false strength(control): 0.90 prompt: a lovely cat holding a sign says 'flux.cpp' negative_prompt: min_cfg: 1.00 cfg_scale: 1.00 guidance: 3.50 clip_skip: -1 width: 512 height: 512 sample_method: euler schedule: default sample_steps: 4 strength(img2img): 0.75 rng: cuda seed: 42 batch_count: 1 vae_tiling: false upscale_repeats: 1 System Info: BLAS = 1 SSE3 = 1 AVX = 1 AVX2 = 1 AVX512 = 0 AVX512_VBMI = 0 AVX512_VNNI = 0 FMA = 1 NEON = 0 ARM_FMA = 0 F16C = 1 FP16_VA = 0 WASM_SIMD = 0 VSX = 0 [DEBUG] stable-diffusion.cpp:166 - Using Vulkan backend ggml_vulkan: Found 1 Vulkan devices: Vulkan0: NVIDIA GeForce RTX 3070 Ti (NVIDIA) | uma: 0 | fp16: 1 | warp size: 32 [INFO ] stable-diffusion.cpp:202 - loading clip_l from './flux/clip_l.safetensors' [INFO ] model.cpp:793 - load ./flux/clip_l.safetensors using safetensors format [DEBUG] model.cpp:861 - init from './flux/clip_l.safetensors' [INFO ] stable-diffusion.cpp:209 - loading t5xxl from 'D./flux/t5xxl_fp8_e4m3fn.safetensors' [WARN ] model.cpp:799 - unknown format D./flux/t5xxl_fp8_e4m3fn.safetensors [WARN ] stable-diffusion.cpp:211 - loading t5xxl from 'D./flux/t5xxl_fp8_e4m3fn.safetensors' failed [INFO ] stable-diffusion.cpp:216 - loading diffusion model from './flux/schnell/flux1-schnell-q8_0.gguf' [INFO ] model.cpp:790 - load ./flux/schnell/flux1-schnell-q8_0.gguf using gguf format [DEBUG] model.cpp:807 - init from './flux/schnell/flux1-schnell-q8_0.gguf' WARNING: Behavior may be unexpected when allocating 0 bytes for ggml_calloc! [INFO ] stable-diffusion.cpp:223 - loading vae from './flux/ae.safetensors' [INFO ] model.cpp:793 - load ./flux/ae.safetensors using safetensors format [DEBUG] model.cpp:861 - init from './flux/ae.safetensors' [INFO ] stable-diffusion.cpp:235 - Version: Flux Schnell [INFO ] stable-diffusion.cpp:266 - Weight type: f16 [INFO ] stable-diffusion.cpp:267 - Conditioner weight type: f16 [INFO ] stable-diffusion.cpp:268 - Diffusion model weight type: q8_0 [INFO ] stable-diffusion.cpp:269 - VAE weight type: f32 [DEBUG] stable-diffusion.cpp:271 - ggml tensor size = 400 bytes [INFO ] stable-diffusion.cpp:310 - set clip_on_cpu to true [INFO ] stable-diffusion.cpp:313 - CLIP: Using CPU backend [DEBUG] clip.hpp:171 - vocab size: 49408 [DEBUG] clip.hpp:182 - trigger word img already in vocab [DEBUG] ggml_extend.hpp:1050 - clip params backend buffer size = 235.06 MB(RAM) (196 tensors) [DEBUG] ggml_extend.hpp:1050 - t5 params backend buffer size = 9083.77 MB(RAM) (219 tensors) ggml_vulkan: Device memory allocation of size 4255727616 failed. ggml_vulkan: vk::Device::allocateMemory: ErrorOutOfDeviceMemory [ERROR] ggml_extend.hpp:1042 - flux alloc params backend buffer failed, num_tensors = 776 [DEBUG] ggml_extend.hpp:1050 - vae params backend buffer size = 94.57 MB(VRAM) (138 tensors) [DEBUG] stable-diffusion.cpp:398 - loading weights [DEBUG] model.cpp:1530 - loading tensors from ./flux/clip_l.safetensors [DEBUG] model.cpp:1530 - loading tensors from ./flux/schnell/flux1-schnell-q8_0.gguf

It stop at here, not sure what is happening

SkutteOleg commented 1 week ago

ggml_vulkan: Device memory allocation of size 4255727616 failed. ggml_vulkan: vk::Device::allocateMemory: ErrorOutOfDeviceMemory [ERROR] ggml_extend.hpp:1042 - flux alloc params backend buffer failed, num_tensors = 776

Perhaps Flux is too big to allocate on Vulkan. Try a quant that is smaller than 4 GB

stereomatchingkiss commented 1 week ago

ggml_vulkan: Device memory allocation of size 4255727616 failed. ggml_vulkan: vk::Device::allocateMemory: ErrorOutOfDeviceMemory [ERROR] ggml_extend.hpp:1042 - flux alloc params backend buffer failed, num_tensors = 776

Perhaps Flux is too big to allocate on Vulkan. Try a quant that is smaller than 4 GB

Tried with q2 and q4, it show error message like " tensor 'text_encoders.t5xxl.encoder.block.1.layer.0.SelfAttention.v.weight' not in model file", details post at here

SkutteOleg commented 1 week ago

--t5xxl D./flux/t5xxl_fp8_e4m3fn.safetensors

You have a typo in your command

stereomatchingkiss commented 1 week ago

--t5xxl D./flux/t5xxl_fp8_e4m3fn.safetensors

You have a typo in your command

Thanks, at last it work for flux1-schnell-q2_k.gguf and flux1-schnell-q4_0.gguf, but do not work for flux1-schnell-q8_0.gguf(out of memory I guess)

[DEBUG] ggml_extend.hpp:1050 - t5 params backend buffer size = 9083.77 MB(RAM) (219 tensors) ggml_vulkan: Device memory allocation of size 4255727616 failed. ggml_vulkan: vk::Device::allocateMemory: ErrorOutOfDeviceMemory [ERROR] ggml_extend.hpp:1042 - flux alloc params backend buffer failed, num_tensors = 776 [DEBUG] ggml_extend.hpp:1050 - vae params backend buffer size = 94.57 MB(VRAM) (138 tensors) [DEBUG] stable-diffusion.cpp:398 - loading weights [DEBUG] model.cpp:1530 - loading tensors from ./flux/clip_l.safetensors [DEBUG] model.cpp:1530 - loading tensors from ./flux/t5xxl_fp8_e4m3fn.safetensors [INFO ] model.cpp:1685 - unknown tensor 'text_encoders.t5xxl.encoder.embed_tokens.weight | f8_e4m3 | 2 [4096, 32128, 1, 1, 1]' in model file [DEBUG] model.cpp:1530 - loading tensors from ./flux/schnell/flux1-schnell-q8_0.gguf

SkutteOleg commented 1 week ago

Should be fixed by https://github.com/leejet/stable-diffusion.cpp/pull/380