Macos cannot find openblas installation

[juhahalme]

I have issue in getting openblas/clblast working on macos in version above 1.10. I have both installed through homebrew. When I start server, i get error "Warning: OpenBLAS library file not found. Non-BLAS library will be used."

I got openblas working in 1.10 with help from previous issue in https://github.com/LostRuins/koboldcpp/issues/26#issuecomment-1500872089 by doing following changes to makefile:

ifdef LLAMA_OPENBLAS CFLAGS += -DGGML_USE_OPENBLAS -I/usr/local/opt/openblas/include LDFLAGS += -L/usr/local/Cellar/openblas/0.3.23/lib -lopenblas

However, this does not work anymore in 1.11, as makefile was changed. I don't have experience in coding, so I don't really know what do i need to change in makefile to identify openblas installation.

[LostRuins]

@juhahalme try pull the latest version and compile... can you tell me what errors you are getting when you run make LLAMA_OPENBLAS=1 ?

Otherwise, If the compile worked correctly, please tell me what files are generated - you should see a couple of *.so files.

[juhahalme]

I'm getting following error at the end of make:

clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [koboldcpp_openblas] Error 1

I see following so files:

• koboldcpp_noavx2.so
• koboldcpp.so

[LostRuins]

The linker is failing when trying to build openblas version. Can you add the verbose flag and find out why? Or check the terminal output. Make sure you have libopenblas installed in an accessible path - add the above -L path flags whenever you see libopenblas inside the makefile

[juhahalme]

With verbose command i get following, which i think is related to this issue:

Finished prerequisites of target file koboldcpp_noavx2'. Must remake targetkoboldcpp_noavx2'. c++ -I. -I./examples -I./include -I./include/CL -Ofast -DNDEBUG -std=c++11 -fPIC -pthread -s -Wno-multichar -pthread ggml_noavx2.o ggml_rwkv.o ggml_v1_noavx2.o expose.o common.o gpttype_adapter.o -shared -o koboldcpp_noavx2.so -framework Accelerate ld: warning: option -s is obsolete and being ignored Successfully remade target file koboldcpp_noavx2'. Considering target filekoboldcpp_openblas'. File koboldcpp_openblas' does not exist. Considering target fileggml_openblas.o'. Pruning file ggml.c'. Pruning fileggml.h'. Finished prerequisites of target file ggml_openblas.o'. Prerequisiteggml.c' is older than target ggml_openblas.o'. Prerequisiteggml.h' is older than target ggml_openblas.o'. No need to remake targetggml_openblas.o'. Pruning file ggml_rwkv.o'. Pruning fileggml_v1.o'. Pruning file expose.o'. Pruning filecommon.o'. Pruning file gpttype_adapter.o'. Finished prerequisites of target filekoboldcpp_openblas'. Must remake target `koboldcpp_openblas'. c++ -I. -I./examples -I./include -I./include/CL -Ofast -DNDEBUG -std=c++11 -fPIC -pthread -s -Wno-multichar -pthread ggml_openblas.o ggml_rwkv.o ggml_v1.o expose.o common.o gpttype_adapter.o -lopenblas -shared -o koboldcpp_openblas.so -framework Accelerate ld: warning: option -s is obsolete and being ignored ld: library not found for -lopenblas clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [koboldcpp_openblas] Error 1

In makefile there are these lines that have libopenblas:

OPENBLAS_BUILD = $(CXX) $(CXXFLAGS) $^ lib/libopenblas.lib -shared -o $@.dll $(LDFLAGS)
OPENBLAS_NOAVX2_BUILD = $(CXX) $(CXXFLAGS) $^ lib/libopenblas.lib -shared -o $@.dll $(LDFLAGS)

I have tried to modify these two lines, but I haven't been able to get it working. Could you give me example how to modify these lines?

EDIT:

Ok, i got it working! Now i get following message when starting server:

Attempting to use OpenBLAS library for faster prompt ingestion. A compatible libopenblas will be required. Initializing dynamic library: koboldcpp_openblas.so

Thank you for the help!

[LostRuins]

Great, glad you got it working. I suppose it was just adding the library paths at the right places.

Please close this issue if all is resolved.

[juhahalme]

Openblas seems to be working now properly, but I cannot get clblast working. I don't know if my system even supports as I have old intel imac with amd 580 gpu. I installed clblast through homebrew and added proper flags to the makefile. However, I don't know what directory should I add for -lOpenCL. I have read that macos doesn't support anymore opencl, so i'm not sure what should I add here. I have tried to install this https://formulae.brew.sh/formula/ocl-icd, and while it doesn't give any errors, it doesn't recognize my GPU and crashes when I try to generate anything. Any ideas on this?

[LostRuins]

Unfortunately I am not familiar with setting up CLBlast on imac.

[dogjamboree]

@juhahalme, what did you do exactly to get this working on mac? I'm having the exact same issue. Could you paste the config lines from your working config? Thanks!

[juhahalme]

@dogjamboree, Firstly, I only got openblas working, but I never got clblast working on any mac I tried (M2 macbook air and iMac 2017 with AMD radeon 580 pro GPU). I think the problem is that opencl was depreciated by apple. What i did to get openblas working was to first install openblas through homebrew. Then I added openblas lib directory before each -lopenblas flag in makefile. For example:

OPENBLAS_BUILD = $(CXX) $(CXXFLAGS) $^ $(ARCH_ADD) -L/opt/homebrew/opt/openblas/lib -lopenblas -shared -o $@.so $(LDFLAGS)

You can find openblas directory with "brew info openblas". I tried also to install clblast through homebrew and linked it's library in same way to the makefile. However, I'm still missing what directory should I add before -lOpenCL flag. I have tried everything and it won't work, so I gave up trying.

[LostRuins]

If you're building on macbook, you should be using Accelerate, which is much faster than openblas on that platform. It should be enabled by default.

[juhahalme]

Oh, I didn't know that. I guess that means that I just select non-blas library from the start menu? It would appear that this option is indeed faster. Thanks for info!

[dogjamboree]

Thanks guys...good to know, all of this.

[dogjamboree]

Just occurred to me that maybe you should update the readme to mention this note about accelerate being faster than openblas for MacOS because the way it stands now (grouped together with linux) it sounds like something you'd want to try and accomplish (which is why I kept trying to make it happen). Just a suggestion ;)