How to enable AVX-512 and use all CPU threads of system

antimatter15 / alpaca.cpp

Locally run an Instruction-Tuned Chat-Style LLM

MIT License

10.25k stars 906 forks source link

How to enable AVX-512 and use all CPU threads of system #66

Open y0n9q1 opened 1 year ago

y0n9q1 commented 1 year ago

Use a Intel CPU with AVX-512 but AVX-512=0 during boot up of the chat as following. So how to enable AVX-512 to speed up? Also my system have 8 thread but seems only 4 are used(CPU usage cap at 50%). So how to config it?

system_info: n_threads = 4 / 8 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 | main: interactive mode on.

Khalilbz commented 1 year ago

Just add -t 8 for more information, run ./chat -h