leejet / stable-diffusion.cpp

Stable Diffusion in pure C/C++
MIT License
2.94k stars 239 forks source link

stable-diffusion.cpp doesn't compile with flash attention enabled #297

Open OkLunixBSD opened 2 weeks ago

OkLunixBSD commented 2 weeks ago

Trying to compile the program with flash attention enabled errors out apparently, the ggml_flash_attn() function was replaced by ggml_flash_attn_ext() in ggml in this pr: https://github.com/ggerganov/ggml/commit/1a815669c255a4294cbf8e4c9010c67f55251ff4 cmake configuration: cmake -B build -DSD_FLASH_ATTN=ON compilation:

$ cmake --build build
[ 15%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml.c.o
[ 15%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml-backend.c.o
[ 23%] Building C object thirdparty/CMakeFiles/zip.dir/zip.c.o
[ 30%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml-alloc.c.o
[ 38%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml-quants.c.o
In file included from /home/kmf/pg/image/flash/thirdparty/zip.c:40:
/home/kmf/pg/image/flash/thirdparty/miniz.h:4988:9: note: ‘#pragma message: Using fopen, ftello, fseeko, stat() etc. path for file I/O - this path may not support large files.’
 4988 | #pragma message(                                                               \
      |         ^~~~~~~
[ 38%] Built target zip
[ 46%] Linking C static library libggml.a
[ 46%] Built target ggml
[ 61%] Building CXX object CMakeFiles/stable-diffusion.dir/upscaler.cpp.o
[ 69%] Building CXX object CMakeFiles/stable-diffusion.dir/model.cpp.o
[ 69%] Building CXX object CMakeFiles/stable-diffusion.dir/stable-diffusion.cpp.o
[ 76%] Building CXX object CMakeFiles/stable-diffusion.dir/util.cpp.o
In file included from /home/kmf/pg/image/flash/esrgan.hpp:4,
                 from /home/kmf/pg/image/flash/upscaler.cpp:1:
/home/kmf/pg/image/flash/ggml_extend.hpp: In function ‘ggml_tensor* ggml_nn_attention(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, bool)’:
/home/kmf/pg/image/flash/ggml_extend.hpp:592:31: error: ‘ggml_flash_attn’ was not declared in this scope; did you mean ‘ggml_flash_attn_ext’?
  592 |     struct ggml_tensor* kqv = ggml_flash_attn(ctx, q, k, v, false);  // [N * n_head, n_token, d_head]
      |                               ^~~~~~~~~~~~~~~
      |                               ggml_flash_attn_ext
In file included from /home/kmf/pg/image/flash/stable-diffusion.cpp:1:
/home/kmf/pg/image/flash/ggml_extend.hpp: In function ‘ggml_tensor* ggml_nn_attention(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, bool)’:
/home/kmf/pg/image/flash/ggml_extend.hpp:592:31: error: ‘ggml_flash_attn’ was not declared in this scope; did you mean ‘ggml_flash_attn_ext’?
  592 |     struct ggml_tensor* kqv = ggml_flash_attn(ctx, q, k, v, false);  // [N * n_head, n_token, d_head]
      |                               ^~~~~~~~~~~~~~~
      |                               ggml_flash_attn_ext
In file included from /home/kmf/pg/image/flash/preprocessing.hpp:4,
                 from /home/kmf/pg/image/flash/util.cpp:13:
/home/kmf/pg/image/flash/ggml_extend.hpp: In function ‘ggml_tensor* ggml_nn_attention(ggml_context*, ggml_tensor*, ggml_tensor*, ggml_tensor*, bool)’:
/home/kmf/pg/image/flash/ggml_extend.hpp:592:31: error: ‘ggml_flash_attn’ was not declared in this scope; did you mean ‘ggml_flash_attn_ext’?
  592 |     struct ggml_tensor* kqv = ggml_flash_attn(ctx, q, k, v, false);  // [N * n_head, n_token, d_head]
      |                               ^~~~~~~~~~~~~~~
      |                               ggml_flash_attn_ext
gmake[2]: *** [CMakeFiles/stable-diffusion.dir/build.make:118: CMakeFiles/stable-diffusion.dir/util.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[2]: *** [CMakeFiles/stable-diffusion.dir/build.make:104: CMakeFiles/stable-diffusion.dir/upscaler.cpp.o] Error 1
gmake[2]: *** [CMakeFiles/stable-diffusion.dir/build.make:90: CMakeFiles/stable-diffusion.dir/stable-diffusion.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:167: CMakeFiles/stable-diffusion.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2