Bug: [Regression] Cannot build with hipblas

[Mushoz]

What happened?

I can no longer build llama.cpp with hipblas enabled. The following dockerfile can be used to reproduce the issue:

FROM rocm/pytorch

ARG ROCM_TARGET_LST=/root/gfx

RUN echo "gfx1100" > /root/gfx

WORKDIR /root/
RUN git clone https://github.com/ggerganov/llama.cpp.git
WORKDIR /root/llama.cpp
RUN make GGML_HIPBLAS=1 -j$(nproc)

The relevant log output seems to be the following:

MK_CPPFLAGS += -DGGML_USE_CPU_AARCH64
make: MK_CPPFLAGS: No such file or directory
make: *** [Makefile:840: ggml/src/ggml-cuda/mmvq.o] Error 127

I bisected the issue, and this was found to be the first bad commit:

1607a5e5b08f4e55f118af3d7de325949d8f1835 is the first bad commit
commit 1607a5e5b08f4e55f118af3d7de325949d8f1835
Author: Charles Xu <charles.xu@arm.com>
Date:   Fri Nov 15 01:28:50 2024 +0100

    backend cpu: add online flow for aarch64 Q4_0 GEMV/GEMM kernels (#9921)

    * backend-cpu: add online flow for aarch64 Q4_0 GEMV/GEMM kernels

    ---------

    Co-authored-by: Diego Devesa <slarengh@gmail.com>

 Makefile                             |   4 +
 ggml/CMakeLists.txt                  |   1 +
 ggml/include/ggml-cpu.h              |   3 +
 ggml/src/ggml-cpu/CMakeLists.txt     |   5 ++
 ggml/src/ggml-cpu/ggml-cpu-aarch64.c | 144 +++++++++++++++++++++++++++++++++++
 ggml/src/ggml-cpu/ggml-cpu-aarch64.h |   3 +
 ggml/src/ggml-cpu/ggml-cpu.c         |  23 +++---
 ggml/src/ggml-cpu/ggml-cpu.cpp       | 106 +++++++++++++++++++++++---
 src/llama.cpp                        |   2 +-
 9 files changed, 271 insertions(+), 20 deletions(-)

Name and Version

It doesn't build, but the first broken version is commit 1607a5e5b08f4e55f118af3d7de325949d8f1835

What operating system are you seeing the problem on?

Linux

Relevant log output

No response

[slaren]

You should be able to build with HIP by using cmake with -DGGML_HIP=ON. The Makefile will likely be removed in the near future.

[David4D]

Same issue with build command make GGML_CUDA=1 with Release/b4082

[FirstTimeEZ]

@Mushoz what happens when you run this

cmake -B build -DGGML_HIP=ON
cmake --build build --config Release