Prerequisites

Please answer the following questions for yourself before submitting an issue.

[x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
[x] I carefully followed the README.md.
[x] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
[x] I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

Running command sudo make LLAMA_OPENBLAS=1 to compile binary in RHEL9.1.

Current Behavior

Command returned an error related to openblas library (see below). Compiling works without 'LLAMA_OPENBLAS=1`' argument though.

Steps to Reproduce

sudo wget https://github.com/LostRuins/koboldcpp/archive/refs/tags/v1.43.tar.gz
sudo tar -xf v1.43.tar.gz
cd koboldcpp-1.43/
sudo yum install openblas make gcc gcc-c++
sudo make LLAMA_OPENBLAS=1

Failure Logs

I llama.cpp build info:
I UNAME_S:  Linux
I UNAME_P:  x86_64
I UNAME_M:  x86_64
I CFLAGS:   -I.              -I./include -I./include/CL -I./otherarch -I./otherarch/tools -Ofast -DNDEBUG -std=c11   -fPIC -DGGML_USE_K_QUANTS -DLOG_DISABLE_LOGS -pthread -s -pthread -march=native -mtune=native
I CXXFLAGS: -I. -I./common -I./include -I./include/CL -I./otherarch -I./otherarch/tools -Ofast -DNDEBUG -std=c++11 -fPIC -DGGML_USE_K_QUANTS -DLOG_DISABLE_LOGS -pthread -s -Wno-multichar -Wno-write-strings -pthread
I LDFLAGS:
I CC:       cc (GCC) 11.3.1 20221121 (Red Hat 11.3.1-4)
I CXX:      g++ (GCC) 11.3.1 20221121 (Red Hat 11.3.1-4)

g++ -I. -I./common -I./include -I./include/CL -I./otherarch -I./otherarch/tools -Ofast -DNDEBUG -std=c++11 -fPIC -DGGML_USE_K_QUANTS -DLOG_DISABLE_LOGS -pthread -s -Wno-multichar -Wno-write-strings -pthread  ggml.o ggml_v2.o ggml_v1.o expose.o common.o gpttype_adapter.o k_quants.o ggml-alloc.o -shared -o koboldcpp_default.so
g++ -I. -I./common -I./include -I./include/CL -I./otherarch -I./otherarch/tools -Ofast -DNDEBUG -std=c++11 -fPIC -DGGML_USE_K_QUANTS -DLOG_DISABLE_LOGS -pthread -s -Wno-multichar -Wno-write-strings -pthread ggml_failsafe.o ggml_v2_failsafe.o ggml_v1_failsafe.o expose.o common.o gpttype_adapter_failsafe.o k_quants_failsafe.o ggml-alloc.o -shared -o koboldcpp_failsafe.so
g++ -I. -I./common -I./include -I./include/CL -I./otherarch -I./otherarch/tools -Ofast -DNDEBUG -std=c++11 -fPIC -DGGML_USE_K_QUANTS -DLOG_DISABLE_LOGS -pthread -s -Wno-multichar -Wno-write-strings -pthread ggml_openblas.o ggml_v2_openblas.o ggml_v1.o expose.o common.o gpttype_adapter.o k_quants.o ggml-alloc.o  -lopenblas -shared -o koboldcpp_openblas.so
/bin/ld: cannot find -lopenblas
collect2: error: ld returned 1 exit status
make: *** [Makefile:435: koboldcpp_openblas] Error 1

Environment and Context

Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.

Physical (or virtual) hardware you are using, e.g. for Linux:

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         45 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  16
  On-line CPU(s) list:   0-15
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Xeon(R) Gold 6142 CPU @ 2.60GHz
    CPU family:          6
    Model:               63
    Thread(s) per core:  1
    Core(s) per socket:  1
    Socket(s):           16
    Stepping:            0
    BogoMIPS:            5199.99
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable no
                         nstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti
                         ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid xsaveopt arat md_clear flush_l1d arch_capabilities
Virtualization features:
  Hypervisor vendor:     VMware
  Virtualization type:   full
Caches (sum of all):
  L1d:                   512 KiB (16 instances)
  L1i:                   512 KiB (16 instances)
  L2:                    16 MiB (16 instances)
  L3:                    352 MiB (16 instances)
NUMA:
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-15
Vulnerabilities:
  Itlb multihit:         KVM: Mitigation: VMX unsupported
  L1tf:                  Mitigation; PTE Inversion
  Mds:                   Mitigation; Clear CPU buffers; SMT Host state unknown
  Meltdown:              Mitigation; PTI
  Mmio stale data:       Mitigation; Clear CPU buffers; SMT Host state unknown
  Retbleed:              Mitigation; IBRS
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; IBRS, IBPB conditional, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
  Srbds:                 Not affected
  Tsx async abort:       Not affected

Operating System, e.g. for Linux:

Linux XXX.domain.XXX 5.14.0-284.30.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Aug 25 09:13:12 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux

SDK version, e.g. for Linux:

$ python 3.9.16
$ GNU Make 4.3
$ g++ (GCC) 11.3.1 20221121 (Red Hat 11.3.1-4)
$ openblas 0.3.21

Looks like openblas was not installed correctly, as the library was not found. Did the package manager install it correctly? Are there other packages with different names that you can try, such as openblas-devel or openblas-static? Make sure the library paths are correct too.

Alternatively, maybe try getting it from here: https://rhel.pkgs.org/7/epel-x86_64/openblas-devel-0.3.3-2.el7.x86_64.rpm.html

Looks like openblas was not installed correctly, as the library was not found. Did the package manager install it correctly? Are there other packages with different names that you can try, such as openblas-devel or openblas-static? Make sure the library paths are correct too.

Alternatively, maybe try getting it from here: https://rhel.pkgs.org/7/epel-x86_64/openblas-devel-0.3.3-2.el7.x86_64.rpm.html

Thanks for your prompt reply. The installation went smoothly running yum install openblas openblas-devel and the libraries are correctly installed:

$ yum list installed
[...]
openblas.x86_64                                                                             0.3.21-2.el9                                                                @rhel-9-for-x86_64-appstream-rpms
openblas-devel.x86_64                                                                       0.3.21-2.el9                                                                @codeready-builder-for-rhel-9-x86_64-rpms
openblas-openmp.x86_64                                                                      0.3.21-2.el9                                                                @rhel-9-for-x86_64-appstream-rpms
openblas-openmp64.x86_64                                                                    0.3.21-2.el9                                                                @codeready-builder-for-rhel-9-x86_64-rpms
openblas-openmp64_.x86_64                                                                   0.3.21-2.el9                                                                @codeready-builder-for-rhel-9-x86_64-rpms
openblas-serial.x86_64                                                                      0.3.21-2.el9                                                                @rhel-9-for-x86_64-appstream-rpms
openblas-serial64.x86_64                                                                    0.3.21-2.el9                                                                @codeready-builder-for-rhel-9-x86_64-rpms
openblas-serial64_.x86_64                                                                   0.3.21-2.el9                                                                @codeready-builder-for-rhel-9-x86_64-rpms
openblas-srpm-macros.noarch                                                                 2-11.el9                                                                    @rhel-9-for-x86_64-appstream-rpms
openblas-threads.x86_64                                                                     0.3.21-2.el9                                                                @codeready-builder-for-rhel-9-x86_64-rpms
openblas-threads64.x86_64                                                                   0.3.21-2.el9                                                                @codeready-builder-for-rhel-9-x86_64-rpms
openblas-threads64_.x86_64                                                                  0.3.21-2.el9                                                                @codeready-builder-for-rhel-9-x86_
[...]

Nevertheless, my openblas packages version (0.3.21) in RHEL9 are older than the one you are suggesting in pgks.org (0.3.3) though your link is for RHEL 7, could that be the problem?

Strangely, when I go to the RHEL9 repository, openblas is not in the list: https://rhel.pkgs.org/9/epel-x86_64/

That's the problem with linux to be honest, there are so many different packages with different names and versions for each distro haha. often times, the only way is to figure it out by trial and error. I have no idea which one is supposed to be the best one - I don't use RHEL myself and have not personally tested it there.

That's the problem with linux to be honest, there are so many different packages with different names and versions for each distro haha. often times, the only way is to figure it out by trial and error. I have no idea which one is supposed to be the best one - I don't use RHEL myself and have not personally tested it there.

Indeed that's pretty messy...

I was able to make it work now, I realized that I was missing package openblas-static.x86_64. Thus, I can confirm that openblas runs fine in RHEL9.1 with the following packages installed:

$ yum list installed | grep openblas
openblas.x86_64                               0.3.21-2.el9                  @rhel-9-for-x86_64-appstream-rpms
openblas-devel.x86_64                         0.3.21-2.el9                  @codeready-builder-for-rhel-9-x86_64-rpms
openblas-openmp.x86_64                        0.3.21-2.el9                  @rhel-9-for-x86_64-appstream-rpms
openblas-openmp64.x86_64                      0.3.21-2.el9                  @codeready-builder-for-rhel-9-x86_64-rpms
openblas-openmp64_.x86_64                     0.3.21-2.el9                  @codeready-builder-for-rhel-9-x86_64-rpms
openblas-serial.x86_64                        0.3.21-2.el9                  @rhel-9-for-x86_64-appstream-rpms
openblas-serial64.x86_64                      0.3.21-2.el9                  @codeready-builder-for-rhel-9-x86_64-rpms
openblas-serial64_.x86_64                     0.3.21-2.el9                  @codeready-builder-for-rhel-9-x86_64-rpms
openblas-srpm-macros.noarch                   2-11.el9                      @rhel-9-for-x86_64-appstream-rpms
openblas-static.x86_64                        0.3.21-2.el9                  @codeready-builder-for-rhel-9-x86_64-rpms
openblas-threads.x86_64                       0.3.21-2.el9                  @codeready-builder-for-rhel-9-x86_64-rpms
openblas-threads64.x86_64                     0.3.21-2.el9                  @codeready-builder-for-rhel-9-x86_64-rpms
openblas-threads64_.x86_64                    0.3.21-2.el9                  @codeready-builder-for-rhel-9-x86_64-rpms

This is what I get now when I run the python koboldcpp.py [PARAMS]:

***
Welcome to KoboldCpp - Version 1.43
Attempting to use OpenBLAS library for faster prompt ingestion. A compatible libopenblas will be required.
Initializing dynamic library: koboldcpp_openblas.so
==========
Namespace(model=None, model_param='models/llama-2-7b.Q4_K_M.gguf', port=5001, port_param=8008, host='', launch=False, lora=None, config=None, threads=7, blasthreads=7, psutil_set_threads=False, highpriority=False, contextsize=2048, blasbatchsize=512, ropeconfig=[0.0, 10000.0], stream=True, smartcontext=False, unbantokens=False, bantokens=None, usemirostat=None, forceversion=0, nommap=False, usemlock=False, noavx2=False, debugmode=0, skiplauncher=False, hordeconfig=None, noblas=False, useclblast=None, usecublas=None, gpulayers=0, tensor_split=None)
==========
Loading model: /home/wichtco/koboldcpp-1.43/models/llama-2-7b.Q4_K_M.gguf
[Threads: 7, BlasThreads: 7, SmartContext: False]

Best, C.

LostRuins / koboldcpp

Error compiling with LLAMA_OPENBLAS=1 in RHEL9 #435