OpenBLAS and CLBlast support

[Fr0d0Beutl1n]

Is there a reason that OpenBLAS and CLBlast are not mentioned in the README?

I compiled it with cmake -DLLAMA_OPENBLAS=1 -DLLAMA_CLBLAST=1 .. && cmake --build . --config Release (on Linux), and it appears to be working as expected.

[cmp-nct]

I am focusing currently cpu + the cuda with cublas variants.

Right now I believe metal, clblast and openblas will work but if I mention it in the readme I am forced to fix problems with them and keep them synchronized with new features. I can't test metal and I had to focus on one variant - I'm most familiar with cuda+cublas. clblast won't react to reserved memory or override-gpu flags, so we already move a bit out of support.

We will likely move gradually out of support for some variants, except if someone keeps updating them to stay compatible with new features added. Given how much better Falcon is than Llama it's well possible people will work on keeping their favorite compilation options running.

[chrisbward]

➜  build git:(master) ✗ cmake -DLLAMA_OPENBLAS=1 -DLLAMA_CLBLAST=1 .. && cmake --build . --config Release
CMake Warning at CMakeLists.txt:239 (find_package):
  By not providing "FindCUDAToolkit.cmake" in CMAKE_MODULE_PATH this project
  has asked CMake to find a package configuration file provided by
  "CUDAToolkit", but CMake did not find one.

  Could not find a package configuration file provided by "CUDAToolkit" with
  any of the following names:

    CUDAToolkitConfig.cmake
    cudatoolkit-config.cmake

  Add the installation prefix of "CUDAToolkit" to CMAKE_PREFIX_PATH or set
  "CUDAToolkit_DIR" to a directory containing one of the above files.  If
  "CUDAToolkit" provides a separate development package or SDK, be sure it
  has been installed.

CMake Warning at CMakeLists.txt:260 (message):
  cuBLAS not found

CMake Warning at CMakeLists.txt:295 (find_package):
  By not providing "FindCLBlast.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "CLBlast", but
  CMake did not find one.

  Could not find a package configuration file provided by "CLBlast" with any
  of the following names:

    CLBlastConfig.cmake
    clblast-config.cmake

  Add the installation prefix of "CLBlast" to CMAKE_PREFIX_PATH or set
  "CLBlast_DIR" to a directory containing one of the above files.  If
  "CLBlast" provides a separate development package or SDK, be sure it has
  been installed.

CMake Warning at CMakeLists.txt:305 (message):
  CLBlast not found

-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Configuring done
-- Generating done
-- Build files have been written to: /home/user/Tools/06_MachineLearning/LLM/ggllm.cpp/build
[  4%] Built target ggml
[  8%] Built target llama
[ 12%] Built target cmpnct_unicode
[ 16%] Built target falcon
[ 18%] Built target ggml_static
[ 20%] Built target BUILD_INFO
[ 25%] Built target test-sampling
[ 29%] Built target test-quantize-perf
[ 33%] Built target test-quantize-fns
[ 35%] Built target falcon_common
[ 37%] Built target common
[ 41%] Built target main
[ 45%] Built target falcon_main
[ 50%] Built target falcon_quantize
[ 54%] Built target quantize
[ 58%] Built target quantize-stats
[ 62%] Built target perplexity
[ 66%] Built target falcon_perplexity
[ 70%] Built target embedding
[ 75%] Built target save-load-state
[ 79%] Built target benchmark
[ 83%] Built target baby-llama
[ 87%] Built target train-text-from-scratch
[ 91%] Built target simple
[ 95%] Built target q8dot
[100%] Built target vdot

I can't seem to get past these errors, always compiled without GPU-support. I have cblast installed, but those two files are not on my system?

CLBlastConfig.cmake
clblast-config.cmake

[Fr0d0Beutl1n]

CLBlast is what gives you GPU support. Cmake can't find the config file CLBlastConfig.cmake that would enable it to compile the program with GPU support. The problem you have, is not with ggllm.cpp. Either, you did not install CLBlast correctly, or you specified a wrong path.

On my system (Arch Linux) CLBlastConfig.cmake is in /usr/lib/cmake/CLBlast, and I don't have clblast-config.cmake. The log message you provided states, that you only need one of those two files, which makes sense, considering that I was able to compile it with just one of them. I assume that they are just different names for the same file.

Maybe this helps? If the installation fails for whatever reason, you could just build CLBlast, and then, when building ggllm.cpp, specify the directory where you built CLBlast with -DCMAKE_INSTALL_PREFIX=/path/to/install/directory.

[cmp-nct]

Forgot to answer: try switching off cublas defines, it looks like cublas is being compiled but you want clblast. -DLLAMA_CUBLAS=0