Getting Errors When following Readme Instruction on ARM server with ubuntu24.04

[sadaqatullah]

27 warnings and 6 errors generated.
gmake[2]: *** [3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/build.make:146: 3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/__/__/__/__/src/ggml-bitnet-lut.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:759: 3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

In the log files I am getting this when running python setup_env.py --hf-repo HF1BitLLM/Llama3-8B-1.58-100B-tokens -q i2_s on linux server with arm CPU.

When I ran grep -i "error" logs/compile.log, I got following list of errors:

(bitnet-cpp) root@BitnetTesting:~/BitNet# grep -i "error" logs/compile.log 
/root/BitNet/3rdparty/llama.cpp/ggml/src/../../../../include/bitnet-lut-kernels.h:12:5: error: use of undeclared identifier 'posix_memalign'
/root/BitNet/3rdparty/llama.cpp/ggml/src/../../../../include/bitnet-lut-kernels.h:20:5: error: use of undeclared identifier 'free'
/root/BitNet/3rdparty/llama.cpp/ggml/src/../../../../include/bitnet-lut-kernels.h:289:5: error: use of undeclared identifier 'memset'
/root/BitNet/3rdparty/llama.cpp/ggml/src/../../../../include/bitnet-lut-kernels.h:423:5: error: use of undeclared identifier 'memset'
/root/BitNet/3rdparty/llama.cpp/ggml/src/../../../../include/bitnet-lut-kernels.h:541:5: error: use of undeclared identifier 'memset'
/root/BitNet/3rdparty/llama.cpp/ggml/src/../../../../include/bitnet-lut-kernels.h:675:5: error: use of undeclared identifier 'memset'
27 warnings and 6 errors generated.
gmake[2]: *** [3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/build.make:146: 3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/__/__/__/__/src/ggml-bitnet-lut.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:759: 3rdparty/llama.cpp/ggml/src/CMakeFiles/ggml.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

[bmerkle]

posix_memalign() is defined in so make sure that this header file can be found and is included via the code.

For me the code looks a bit strange as #if defined(GGML_BITNET_X86_TL2) refers to X86 and not ARM... ?!

But we have to ask one of the authors about details. Not sure if the matrix https://github.com/microsoft/BitNet?tab=readme-ov-file#supported-models is maybe not correct for Llama3-8B-1.58-100B-tokens and ARM platform

[vahid83]

I am also facing the exact same issue on UB22.04 on aarch64. I can also confirm that the path to is available to clang.

[MrEcco]

I found what's wrong in there and prepared a hotfix. I'm not a maintainer, thus it might take a while to approve it.

https://github.com/microsoft/BitNet/pull/79

[rebroad]

Same problem on Raspbian bookworm

[potassiummmm]

79