Open stefangweichinger opened 10 months ago
Try to install libabsl-dev.
On ubuntu I do apt install libabsl-dev
and it help
I also got this bug following the same https://localai.io/howtos/easy-setup-docker-gpu/ ... unforutunately didnt find this 'apt install libabsl-dev' and am in the process of a new pull/build, will give it a try assuming the issue persists. Could be missing the package on the Quay build?
No luck with the apt install libabsl-dev command, think the GPU installation instructions are borked for the moment.
Pretty sure the bug spawns from instructions: image: quay.io/go-skynet/local-ai:master-cublas-cuda12 something was migrated between packages? Im attempting to follow the How To https://localai.io/howtos/easy-setup-docker-gpu/ with using image instead: quay.io/go-skynet/local-ai:latest (its using cublas/cuda during its build anyway now) will let know how it goes
No, that was not the issue... Man this rabbithole has got me on death spiral... depressed even. My ineptitude screaming at every docker recompose, compile. Docker has always been weak in my IT brain, I know I shouldn't have had to download these packages this many times over again... installing with cublas... Ive got be missing an important memo here... Before I checked cuda boxes the docker seemed to be installing with cuda libraries on its own... Im going to try a no frills install and check if cuda acceleration works, then work within the working base package to try to get cuda libraries upgraded?
@nbollman It's not just you - I'm also having the same problem attempting to compile the image with documented defaults.
same here. fyi: i got the same error when try installing/building locally. So it it is not docker/docker-compose specific..
EDIT: on local build, i was able to solve the absl error message, but after that almost the same error followed, but this time it is protobuf. and I couldn't solve the latter unit now.
@mounta11n how did you resolve absl with local build?
@nbollman I was able to make some progress by:
Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!
_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.
Don't engage in conversation with me, I don't support (yet) replying!
The issue seems to be related to the missing package configuration file for "absl" when trying to build the grpc-server. The error message suggests that the package configuration file "abslConfig.cmake" or "absl-config.cmake" is not found.
One possible solution is to install the "absl" package and its configuration files. You can try installing it using the package manager of your Linux distribution. For example, on Fedora, you can use the following command to install it:
sudo dnf install python3-absl-py.noarch
If the package is already installed, you can try reinstalling it to ensure that the configuration files are installed correctly.
If the package is not available in your package manager, you can try building and installing it from source. You can download the source code from the official GitHub repository: https://github.com/abseil/abseil-cpp.
After installing the "absl" package, try running the build command again to see if the issue is resolved.
Sources:
Issue persists: macOS Sonoma 14, M2
Fixed by installing these depts:
brew install grpc protobuf abseil
Instead of trying to compile and run LocalAI directly on the host, I utilized a prebuilt docker image: quay.io/go-skynet/local-ai:v1.30.0-cublas-cuda12
. Here's the docker run command I used:
docker run -d --name api \
--gpus '"device=0"' \
--restart=always \
-p 8080:8080 \
-e GALLERIES='[{"name":"model-gallery", "url":"github:go-skynet/model-gallery/index.yaml"}, {"url": "github:go-skynet/model-gallery/huggingface.yaml","name":"huggingface"}]' \
-e DEBUG=true \
-e MODELS_PATH=/models \
-e THREADS=8 \
-e BUILD_TYPE=cublas \
-e REBUILD=true \
-e CMAKE_ARGS="-DLLAMA_CUBLAS=on -DLLAMA_F16C=OFF -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_AVX=OFF -DLLAMA_FMA=OFF" \
-e FORCE_CMAKE=1 \
-v $PWD/models:/models \
-t quay.io/go-skynet/local-ai:v1.30.0-cublas-cuda12 \
/usr/bin/local-ai
Here's a breakdown of the notable changes:
--gpus '"device=0"'
option.BUILD_TYPE=cublas
and REBUILD=true
environment variables.-DLLAMA_CUBLAS=on
within CMAKE_ARGS
.FORCE_CMAKE=1
to ensure CMake processes run.By utilizing the above Docker run command, I was able to get LocalAI running successfully with CUDA support.
I hope this helps! Let me know if you face any other challenges.
Hi there, I had the same issue. I fixed by compiling the gRPC locally and installing to a local directory, called ${HOME}/installed_packages. Then I passed a bunch of variables to the CMake to let it find the gRPC, absl, protobuf, etc. Then I ran:
export INSTALLED_PACKAGES="${HOME}/installed_packages/linux_x86_64"
export INSTALLED_LIB_CMAKE="${INSTALLED_PACKAGES}/lib/cmake"
export ADDED_CMAKE_ARGS="-Dabsl_DIR=${INSTALLED_LIB_CMAKE}/absl"
export ADDED_CMAKE_ARGS="${ADDED_CMAKE_ARGS} -DProtobuf_DIR=${INSTALLED_LIB_CMAKE}/protobuf"
export ADDED_CMAKE_ARGS="${ADDED_CMAKE_ARGS} -Dutf8_range_DIR=${INSTALLED_LIB_CMAKE}/utf8_range"
export ADDED_CMAKE_ARGS="${ADDED_CMAKE_ARGS} -DgRPC_DIR=${INSTALLED_LIB_CMAKE}/grpc"
export ADDED_CMAKE_ARGS="${ADDED_CMAKE_ARGS} -DCMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES=${INSTALLED_PACKAGES}/include"
export _PROTOBUF_PROTOC=${INSTALLED_PACKAGES}/bin/protoc
export _GRPC_CPP_PLUGIN_EXECUTABLE=${INSTALLED_PACKAGES}/bin/grpc_cpp_plugin
export PATH=${PATH}:${INSTALLED_PACKAGES}/bin
CMAKE_ARGS="${ADDED_CMAKE_ARGS}" make build
If it can be useful, I can create a separate script with all the commands and create a PR containing it.
I am also now hanging on the protobuf error... anyone solved it (not on mac)?
Hi there,
I submitted the PR #1232 to solve it. I am waiting for feedback: @mudler , could you please review it, since it is my first PR in this project, thank you.
I have seen that the build now fails. As soon as I reach home, I try to understand why.
The issue was fixed. With the code in the master branch, it is possible to build gRPC locally by running the command:
BUILD_GRPC_FOR_BACKEND_LLAMA=ON make build
I think it is possible to close the issue.
The issue was fixed. With the code in the master branch, it is possible to build gRPC locally by running the command:
BUILD_GRPC_FOR_BACKEND_LLAMA=ON make build
I think it is possible to close the issue.
For clarification for other noobs like me: that means adding BUILD_GRPC_FOR_BACKEND_LLAMA=ON
in the .env and not in CMAKE_ARGS or elsewhere
The issue was fixed. With the code in the master branch, it is possible to build gRPC locally by running the command:
BUILD_GRPC_FOR_BACKEND_LLAMA=ON make build
I think it is possible to close the issue.For clarification for other noobs like me: that means adding
BUILD_GRPC_FOR_BACKEND_LLAMA=ON
in the .env and not in CMAKE_ARGS or elsewhere
CMAKE_ARGS="-DLLAMA_AVX512=ON" BUILD_GRPC_FOR_BACKEND_LLAMA=ON make BUILD_TYPE=clblas build
results here in some error on current master branch (commit 562ac62f59b55c831207a908f07144473c2c4599 (HEAD -> master, origin/master, origin/HEAD)
)
gmake[4]: *** [examples/grpc-server/CMakeFiles/hw_grpc_proto.dir/build.make:89: examples/grpc-server/CMakeFiles/hw_grpc_proto.dir/backend.grpc.pb.cc.o] Error 1
gmake[4]: Leaving directory '/home/m/git/LocalAI/backend/cpp/llama/llama.cpp/build'
gmake[3]: *** [CMakeFiles/Makefile2:2492: examples/grpc-server/CMakeFiles/hw_grpc_proto.dir/all] Error 2
gmake[3]: Leaving directory '/home/m/git/LocalAI/backend/cpp/llama/llama.cpp/build'
gmake[2]: *** [Makefile:146: all] Error 2
gmake[2]: Leaving directory '/home/m/git/LocalAI/backend/cpp/llama/llama.cpp/build'
make[1]: *** [Makefile:43: grpc-server] Error 2
make[1]: Leaving directory '/home/m/git/LocalAI/backend/cpp/llama'
make: *** [Makefile:428: backend/cpp/llama/grpc-server] Error 2
This error is reproducible in the official container for 1.40.0-ffmpeg at least.
I attach screenshot.
@EchedeyLR can you share your .env ?
Is exactly the default one at https://github.com/mudler/LocalAI/blob/v1.40.0/.env with GO_TAGS and REBUILD uncommented.
BUILD_GRPC_FOR_BACKEND_LLAMA=ON
If this is needed, why was not included in the example .env
file in the release? I also don't see mention of it outside the issues in the clear documentation.
AFAIK it's not a fix but a workaround while the team has figured out why this started happening.
Personnaly here's what I needed to add at the end of my .env:
CMAKE_ARGS="-DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_FMA=OFF -DLLAMA_CUBLAS=on"
BUILD_GRPC_FOR_BACKEND_LLAMA=ON
FORCE_CMAKE=1
(I also set REBUILD=true and BUILD_TYPE=cublas)
Would this work in the container image?
I am also worried about these fixed CMAKE_ARGS. OpenBLAS did not build for me in go-llama since several months ago, just a version stopped working and I thought that was related to instruction set, as in other computer everything worked so far.
I thought this was self-detected and not hard coded, this just makes it difficult for people to run it in non-exactly modern computers (and no, I am not speaking about something from 2008 or so, I imply 2014 even).
CMAKE_ARGS="-DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_FMA=OFF" BUILD_GRPC_FOR_BACKEND_LLAMA=ON make BUILD_TYPE=clblas build
is working in v1.30.0, but not in v1.40.0 nor on current master (274ace289823a8bacb7b4987b5c961b62d5eee99).
BUILD_GRPC_FOR_BACKEND_LLAMA=ON
If this is needed, why was not included in the example
.env
file in the release? I also don't see mention of it outside the issues in the clear documentation.
Is this needed in the docs? Please ping me if so
We need to confirm this is the issue yet. I am yet to try that solution. Other user reported it as nonworking
My problem was slightly different in that it was giving me errors for absl::check, absl::log, etc.
It turns out Conda installs its own libraries and binaries, which also meant it had its own incompatible version of absl.
I completely uninstalled Anaconda3, then I cloned grpc with
git clone --recurse-submodules -b v1.58.0 --depth 1 --shallow-submodules https://github.com/grpc/grpc
which I got from the Dockerfile
I built and installed abseil-cpp, protobuf (inside grpc/third_party) and grpc from the folder above into /usr/local, after all libraries were installed I built LocalAI as usual.
Here are my build args for LocalAI:
CMAKE_ARGS="-DLLAMA_CUBLAS=ON -DLLAMA_F16C=OFF -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_AVX=OFF -DLLAMA_FMA=OFF" make BUILD_TYPE=cublas GO_TAGS=stablediffusion,tts CUDACXX=/usr/local/cuda/bin/nvcc build
Building as I type this, seems to be ok so far
Ubuntu Server 22.04, Xeon X5650
Just a little report, coming from #1386 using BUILD_GRPC_FOR_BACKEND_LLAMA=ON
for a local compile did the trick. cmake did not find my installed libproto* from debian.
Notable comment there from @B4ckslash
AFAIK this is because not every distribution (Fedora, Debian and derivatives among them) builds Protobuf with CMake, which leads to the .cmake files missing for those distros, so CMake cannot find them when building llama.cpp.
The issue was fixed. With the code in the master branch, it is possible to build gRPC locally by running the command:
BUILD_GRPC_FOR_BACKEND_LLAMA=ON make build
I think it is possible to close the issue.
Thank You so much this worked in my Ubuntu WSL make build
issue
My problem was slightly different in that it was giving me errors for absl::check, absl::log, etc.
It turns out Conda installs its own libraries and binaries, which also meant it had its own incompatible version of absl.
I completely uninstalled Anaconda3, then I cloned grpc with
git clone --recurse-submodules -b v1.58.0 --depth 1 --shallow-submodules https://github.com/grpc/grpc
which I got from the DockerfileI built and installed abseil-cpp, protobuf (inside grpc/third_party) and grpc from the folder above into /usr/local, after all libraries were installed I built LocalAI as usual.
Here are my build args for LocalAI:
CMAKE_ARGS="-DLLAMA_CUBLAS=ON -DLLAMA_F16C=OFF -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_AVX=OFF -DLLAMA_FMA=OFF" make BUILD_TYPE=cublas GO_TAGS=stablediffusion,tts CUDACXX=/usr/local/cuda/bin/nvcc build
Building as I type this, seems to be ok so far
Ubuntu Server 22.04, Xeon X5650
Could you provide a script how to do it? I am facing same build problems
i got everything fixeed "solution" it was just i completed removed that ubuntu WSL and got a new Debian WSL which is very much better than it !! i just download the release "local-ai-avx-Linux-x86_64" and ran it with a .sh script.
localai.sh
#!/bin/sh
export GALLERIES='[{"url": "github:go-skynet/model-gallery/huggingface.yaml","name":"huggingface"}]'
export CORS=true
export ADDRESS=localhost:4040
export DEBUG=true
export PARALLEL_REQUESTS=true
# export WATCHDOG_IDLE=true
export THREADS=18
./local-ai-avx-Linux-x86_64
i am using the model : thebloke__wizardlm-13b-v1-0-uncensored-superhot-8k-ggml__wizardlm-13b-v1.0-superhot-8k.ggmlv3.q4_k_m.bin
, only tested with this model for now.
It works like charm 🪄 🌟
LocalAI version:
45370c212bbc379f65f2c77560958acc24877fba
Environment, CPU architecture, OS, and Version:
Linux fedora 6.5.6-300.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Oct 6 19:57:21 UTC 2023 x86_64 GNU/Linux
Describe the bug
After failures with CUDA and docker in https://github.com/go-skynet/LocalAI/issues/1178
I try to compile and run LocalAI directly on the host:
make BUILD_TYPE=cublas build
To Reproduce
make BUILD_TYPE=cublas build
Expected behavior
Successful build, binaries running with CUDA support
Logs
Additional context
I also tried
CMAKE_ARGS="-DLLAMA_AVX512=OFF" make BUILD_TYPE=cublas build
because my CPU doesn't support AVX512.Maybe interesting:
But the error message seems more like something is missing than it's the wrong gcc.
EDIT: search for absl, found and installed
sudo dnf install python3-absl-py.noarch
... doesn't help.