Closed bearn01d closed 10 months ago
Thanks for your feedback and the detailed report. You are right, we accidently apply GCC avx512 target options to the whole compile unit. We have located the problem, and it will be fixed in a few days.
Thank you for your feedback!
Quantization of Mistral fails with an illegal instruction for me.
gdb disass
shows the usage of AVX-512 instructions likevcvtusi2sd
. I have built from source as described here: https://github.com/intel/intel-extension-for-transformers/tree/ef2c4793c75d4950bfa36f1c1b7c2458bb3bb95a/intel_extension_for_transformers/llm/runtime/graph#how-to-use-python-script.Edit: I have the confirmation that with an AMD Ryzen 4750U, the problem does not occur (Ubuntu 20.04, gcc-10), however the model output is garbage (which however has a different reason probably). Besides, using the i7-1365U, Fedora 39 with gcc-13 in Docker results in the same problem. So I'd be suspicious that some of the Intel-specific optimizations might be the root cause.
The setup is as follows: