Open wallacezq opened 1 month ago
Hi, flash attention is not supported in llama.cpp with IPEX-LLM. We added optimized attention for sycl backend, and it is automatically turned on if the provided model is supported.
Hi, flash attention is not supported in llama.cpp with IPEX-LLM. We added optimized attention for sycl backend, and it is automatically turned on if the provided model is supported.
I see. Can i know how would i know if a model can support optimized attention for sycl backend ?
Here is a list we already verified:
Hi, i encounter the following error message trying to enable flash attention when running the command below. Can i know is flash attention supported ?
command: ./main -m $model -n 128 --prompt "${prompt_1024_128}" -t 8 -e -ngl 999 --color --ctx-size 1024 --no-mmap --temp 0 -fa
see truncated log below: