search

ikawrakow / ik_llama.cpp

llama.cpp fork with additional SOTA quants and improved performance
MIT License
89 stars 6 forks source link

Bug: FA fails when processing prompt lengths that are not a multiple of 8 #34

Closed ikawrakow closed 2 months ago

ikawrakow commented 2 months ago

What happened?

Assert 

iqk_mul_mat.cpp:6163: GGML_ASSERT(S[j] > 0) failed

Name and Version

version: 3408 (57808fd4)

What operating system are you seeing the problem on?

Linux

Relevant log output

Noticed with Gemma2-2b