search

iangitonga / tinyllama.cpp

A C++ implementation of tinyllama inference on CPU.
MIT License
5 stars 1 forks source link

A compiler error #3

Closed Xuquansheng closed 6 months ago

Xuquansheng commented 6 months ago

Hello, i can pass "g++ -std=c++17 -O3 -ffast-math -mavx -mf16c minchatgpt.cpp -o minchatgpt" for your minchatgpt project. When i ran "g++ -std=c++17 -O3 -fopenmp -ffast-math -mavx -mf16c tinyllama.cpp -o tinyllama", i got the error below,

In file included from gten/modules.cpp:6, from gten/gten.h:8, from tinyllama.cpp:1: gten/ops.h: In function ‘float gten::ops::vec_dot_product_q8(const gten::Q8Block, const gten::Q8Block, int)’: gten/ops.h:258:33: error: ‘_mm_loadu_si64’ was not declared in this scope const m128i a00 = _mm_loadu_si64(b0_data); ^~~~~~ gten/ops.h:258:33: note: suggested alternative: ‘_mm_loadl_epi64’ const m128i a00 = _mm_loadu_si64(b0_data); ^~~~~~ _mm_loadl_epi64 gten/ops.h: In function ‘float gten::ops::vec_dot_product_q8_q4(const gten::Q8Block, const gten::Q4Block, int)’: gten/ops.h:340:29: error: ‘_mm_loadu_si64’ was not declared in this scope const m128i a00 = _mm_loadu_si64(a0_data); ^~~~~~ gten/ops.h:340:29: note: suggested alternative: ‘_mm_loadl_epi64’ const m128i a00 = _mm_loadu_si64(a0_data); ^~~~~~ _mm_loadl_epi64

iangitonga commented 6 months ago

Hi, you should only use -mavx and -mf16c compiler options if your CPU supports AVX and F16C instructions. If you are on linux, type lscpu into your terminal and in the Flags section check if avx, sse2 and f16c are listed.

Also, I found that prior versions to GCC 9.0 have some intrinsics missing as commented here. Also ensure you are using GCC version 9.0 or later.

Xuquansheng commented 6 months ago

Thanks. iangitonga, I passed with "g++ -std=c++17 -O3 -fopenmp -mavx -mf16c tinyllama.cpp -o tinyllama", it's low version compiler issue.