Open FranzKafkaYu opened 1 month ago
jeroen-mostert
commented
1 month ago Per https://github.com/ggerganov/llama.cpp/pull/8006, these options have changed names. Use GGML_BLAS and GGML_BLAS_VENDOR instead. The build guide has been updated to document this, so if you came across old documentation, please point it out so it can be updated.
FranzKafkaYu
commented
1 month ago oppps,it's my fault,not a issue.
FranzKafkaYu
commented
1 month ago @jeroen-mostert I have updated this issue，can you help me? What is the right way to build OpenBLAS in Ubuntu 22?
jeroen-mostert
commented
1 month ago I don't have an Ubuntu install at the ready. Check where the package installs the headers and pass the -DBLAS_INCLUDE_DIRS option if necessary (for example, you might need /usr/include/openblas).
smcnally
commented
3 weeks ago On Ubuntu 22.04, I started having build issues this week with make. Running make LLAMA_CUDA=1 or make GGML_CUDA=1 failed with multiple Makefile errors. While reviewing the Makefile, I recloned the repo into a clean subdir, ran make GGML_CUDA=1 again and successfully built functioning binaries.
I'll build OpenBLAS on a clean 22.04 instance without CUDA installed if that helps you troubleshoot. First please let us know here if following Jeroen's comment and these updated build instructions works in your current clone or a new, clean one. e.g. use
-DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS
instead of
-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS
and pass the -DBLAS_INCLUDE_DIRS options as appropriate for your environment.
FranzKafkaYu
commented
3 weeks ago On Ubuntu 22.04, I started having build issues this week with
make. Runningmake LLAMA_CUDA=1ormake GGML_CUDA=1failed with multipleMakefileerrors. While reviewing the Makefile, I recloned the repo into a clean subdir, ranmake GGML_CUDA=1again and successfully built functioning binaries.I'll build OpenBLAS on a clean 22.04 instance without CUDA installed if that helps you troubleshoot. First please let us know here if following Jeroen's comment and these updated build instructions works in your current clone or a new, clean one. e.g. use
-DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLASinstead of
-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLASand pass the
-DBLAS_INCLUDE_DIRSoptions as appropriate for your environment.
yes,I am using -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS flags but the Problem is that the build system can't locate OpenBLAS related libraries and headers.
I am using Ubuntu 22,and I used apt to install OpenBLAS:
sudo apt install libopenblas-dev
sudo apt install libopenblas-base
sudo apt install libopenblas64-dev while when I use dpkg to check what have been installed：
$dpkg -L libopenblas-dev
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/libopenblas-dev
/usr/share/doc/libopenblas-dev/BACKERS.md
/usr/share/doc/libopenblas-dev/CONTRIBUTORS.md.gz
/usr/share/doc/libopenblas-dev/README.md.gz
/usr/share/doc/libopenblas-dev/USAGE.md.gz
/usr/share/doc/libopenblas-dev/copyright
/usr/share/lintian
/usr/share/lintian/overrides
/usr/share/lintian/overrides/libopenblas-dev
/usr/share/doc/libopenblas-dev/changelog.Debian.gzthere are no headers nor libraries. And I checked my system,found that there are some libraries related OpenBLAS:
/usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
/usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3
/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so these libraries are coming from libopenblas0-pthread
but I still can't find related headers:
dpkg -L libopenblas0-pthread
/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/openblas-pthread
/usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
/usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3
/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so
/usr/share
/usr/share/doc
/usr/share/doc/libopenblas0-pthread
/usr/share/doc/libopenblas0-pthread/changelog.Debian.gz
/usr/share/doc/libopenblas0-pthread/copyright
/usr/share/lintian
/usr/share/lintian/overrides
/usr/share/lintian/overrides/libopenblas0-pthread
/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblas.so.0 Append:
Although I found cblas.h in my system:
libopenblas64-pthread-dev:amd64: /usr/include/x86_64-linux-gnu/openblas64-pthread/cblas.h
libopenblas-pthread-dev:amd64: /usr/include/x86_64-linux-gnu/openblas-pthread/cblas.h and set env:
export BLAS_INCLUDE_DIRS=/usr/include/x86_64-linux-gnu/openblas64-pthread it doesn's work,same error here. so what is the right way to build with OpenBLAS?any detailed guidences ？
jeroen-mostert
commented
3 weeks ago I tested it. On a fresh install of Ubuntu 22.04, you need:
sudo apt install build-essential cmake pkg-config libopenblas-devAfter this:
rm -rf ~/llama.cpp.build ; cmake -B ~/llama.cpp.build -S ~/llama.cpp -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS
cd ~/llama.cpp.build
make -jNo further tweaks should be necessary.
Note that using a separate build dir and completely removing it before invoking cmake is highly recommended, because cmake will otherwise leave cruft that can cause subsequent runs to fail due to inappropriately cached results (and will also interfere with git). I suspect this might be what's tripping up your current build.
As an aside, the distro-supplied OpenBLAS will likely not be optimized for the architecture you're running on, so another option is to build OpenBLAS from source (this is easy since it has no dependencies other than what's already included in build-essential) and do a sudo make install PREFIX=/usr/local. cmake will then find and prefer this version. However, be aware that there is no make uninstall, so removing this build cleanly requires manual action.
FranzKafkaYu
commented
3 weeks ago Note that using a separate build dir and completely removing it before invoking
cmakeis highly recommended, becausecmakewill otherwise leave cruft that can cause subsequent runs to fail due to inappropriately cached results (and will also interfere with git). I suspect this might be what's tripping up your current build.
Thank u very much sir,I found that I can compile llama.cpp with OpenBLAS for Linux x86_64 in my Ubuntu,but When I execute cross-platform compile for Android arm64v8 it still report can't find OpenBLAS,details:
$ cmake -B ~/llama.cpp.build.android -S ~/llama.cpp -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=latest -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod
-- Using latest available ANDROID_PLATFORM: 33.
-- The C compiler identification is Clang 14.0.7
-- The CXX compiler identification is Clang 14.0.7
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/franzkafka/Desktop/android/ndk/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/franzkafka/Desktop/android/ndk/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.34.1")
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Found OpenMP_C: -fopenmp=libomp (found version "5.0")
-- Found OpenMP_CXX: -fopenmp=libomp (found version "5.0")
-- Found OpenMP: TRUE (found version "5.0")
-- OpenMP found
-- Could NOT find BLAS (missing: BLAS_LIBRARIES)
CMake Warning at ggml/src/CMakeLists.txt:250 (message):
BLAS not found, please refer to
https://cmake.org/cmake/help/latest/module/FindBLAS.html#blas-lapack-vendors
to set correct GGML_BLAS_VENDOR
-- Using llamafile
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- ARM detected
-- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- Configuring done
-- Generating done
-- Build files have been written to: /home/franzkafka/llama.cpp.build.android Ah, I see. You kinda sorta mentioned this in your first post through the commands, but it's easy to miss. This issue is misnamed; it's not about building with OpenBLAS on Ubuntu 22.04, your issue is specifically about building for Android using OpenBLAS. This is a completely different scenario since obviously we can't use the system's OpenBLAS installation, you need to build OpenBLAS for Android separately and make it available to CMake. This is not specific to llama.cpp, nor even specific to Ubuntu.
I have zero experience building anything for Android and I won't be using this issue as an opportunity to learn. The linked instructions may be out of date and need some tweaking. Then, if you have a build, making things available to CMake is its own challenge. After that, of course, you also have to get OpenBLAS on the Android system, since it's not there by default. Static linking might help there, but last time I tried it (in a different context) CMake kind of insisted on a dynamic link to the BLAS libs even when building GGML statically, so that's yet another hurdle to clear.
Consider if you really need BLAS; if not just building with CPU backend support should be sufficient to get things working on Android. Note that our Android build instructions do not mention using any BLAS implementation; I don't know how much, if any, acceleration is to be expected from using OpenBLAS instead, it may well be a regression.
What happened?
I have followed the guide and install OpenBLAS first：
and I used the command to compile:
and I see these warnings:
I checked CMakeCache.txt and found that the OpenBLAS library didn't get the right path.
Name and Version
version： commit 97bdd26eee11fe109dec00de75690ceef61c03f2 (HEAD, tag: b3400)
What operating system are you seeing the problem on?
No response
Relevant log output
No response