Open nibiru5 opened 1 year ago
Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!
_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.
Don't engage in conversation with me, I don't support (yet) replying!
Sorry for the issue you are facing while building LocalAI on WSL Ubuntu via VSCode. I can understand that it might be frustrating, but I'll try my best to help you resolve this issue. From the information provided, it seems like there is an error while building the llama.cpp/ggml-cuda.o file during the build process. This could be due to several reasons such as incorrect CMake configuration or missing dependencies. To troubleshoot this issue, you can try the following steps:
Sources:
Here is the output I get after running: make VERBOSE=1 BUILD_TYPE=cublas build
git clone --recurse-submodules https://github.com/go-skynet/go-llama.cpp go-llama
Cloning into 'go-llama'...
remote: Enumerating objects: 506, done.
remote: Counting objects: 100% (214/214), done.
remote: Compressing objects: 100% (74/74), done.
remote: Total 506 (delta 159), reused 152 (delta 138), pack-reused 292
Receiving objects: 100% (506/506), 147.49 KiB | 3.51 MiB/s, done.
Resolving deltas: 100% (284/284), done.
Submodule 'llama.cpp' (https://github.com/ggerganov/llama.cpp) registered for path 'llama.cpp'
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-llama/llama.cpp'...
remote: Enumerating objects: 5722, done.
remote: Total 5722 (delta 0), reused 0 (delta 0), pack-reused 5722
Receiving objects: 100% (5722/5722), 4.45 MiB | 14.16 MiB/s, done.
Resolving deltas: 100% (3937/3937), done.
Submodule path 'llama.cpp': checked out '84e09a7d8bc4ab6d658b5cd81295ac0add60be78'
cd go-llama && git checkout -b build f3a6ee0ef53d667f110d28fcf9b808bdca741c07 && git submodule update --init --recursive --depth 1
M llama.cpp
Switched to a new branch 'build'
Submodule path 'llama.cpp': checked out '294f424554c1599784ac9962462fc39ace92d8a5'
git clone --recurse-submodules https://github.com/mudler/go-ggllm.cpp go-ggllm
Cloning into 'go-ggllm'...
remote: Enumerating objects: 55, done.
remote: Counting objects: 100% (55/55), done.
remote: Compressing objects: 100% (38/38), done.
remote: Total 55 (delta 20), reused 43 (delta 12), pack-reused 0
Unpacking objects: 100% (55/55), 25.55 KiB | 39.00 KiB/s, done.
Submodule 'ggllm.cpp' (https://github.com/cmp-nct/ggllm.cpp) registered for path 'ggllm.cpp'
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-ggllm/ggllm.cpp'...
remote: Enumerating objects: 4255, done.
remote: Total 4255 (delta 0), reused 0 (delta 0), pack-reused 4255
Receiving objects: 100% (4255/4255), 107.55 MiB | 17.83 MiB/s, done.
Resolving deltas: 100% (2842/2842), done.
Submodule path 'ggllm.cpp': checked out 'c12b2d65f732a0d8846db2244e070f0f3e73505c'
cd go-ggllm && git checkout -b build 862477d16eefb0805261c19c9b0d053e3b2b684b && git submodule update --init --recursive --depth 1
Switched to a new branch 'build'
git clone --recurse-submodules https://github.com/go-skynet/go-ggml-transformers.cpp go-ggml-transformers
Cloning into 'go-ggml-transformers'...
remote: Enumerating objects: 262, done.
remote: Counting objects: 100% (46/46), done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 262 (delta 36), reused 28 (delta 28), pack-reused 216
Receiving objects: 100% (262/262), 108.41 KiB | 3.19 MiB/s, done.
Resolving deltas: 100% (159/159), done.
Submodule 'ggml.cpp' (https://github.com/ggerganov/ggml) registered for path 'ggml.cpp'
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-ggml-transformers/ggml.cpp'...
remote: Enumerating objects: 2963, done.
remote: Counting objects: 100% (962/962), done.
remote: Compressing objects: 100% (136/136), done.
remote: Total 2963 (delta 872), reused 858 (delta 814), pack-reused 2001
Receiving objects: 100% (2963/2963), 4.72 MiB | 7.22 MiB/s, done.
Resolving deltas: 100% (1955/1955), done.
Submodule path 'ggml.cpp': checked out 'ee1b3727e60403012dd2b57d35b60558f4db66d8'
cd go-ggml-transformers && git checkout -b build && git submodule update --init --recursive --depth 1
Switched to a new branch 'build'
git clone --recurse-submodules https://github.com/nomic-ai/gpt4all gpt4all
Cloning into 'gpt4all'...
remote: Enumerating objects: 9369, done.
remote: Counting objects: 100% (2720/2720), done.
remote: Compressing objects: 100% (478/478), done.
remote: Total 9369 (delta 2359), reused 2454 (delta 2229), pack-reused 6649
Receiving objects: 100% (9369/9369), 12.94 MiB | 14.19 MiB/s, done.
Resolving deltas: 100% (6215/6215), done.
Updating files: 100% (350/350), done.
Submodule 'llama.cpp-230511' (https://github.com/nomic-ai/llama.cpp) registered for path 'gpt4all-backend/llama.cpp-230511'
Submodule 'llama.cpp-230519' (https://github.com/ggerganov/llama.cpp.git) registered for path 'gpt4all-backend/llama.cpp-230519'
Submodule 'llama.cpp-mainline' (https://github.com/nomic-ai/llama.cpp.git) registered for path 'gpt4all-backend/llama.cpp-mainline'
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/gpt4all/gpt4all-backend/llama.cpp-230511'...
remote: Enumerating objects: 3643, done.
remote: Total 3643 (delta 0), reused 0 (delta 0), pack-reused 3643
Receiving objects: 100% (3643/3643), 3.34 MiB | 8.43 MiB/s, done.
Resolving deltas: 100% (2436/2436), done.
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/gpt4all/gpt4all-backend/llama.cpp-230519'...
remote: Enumerating objects: 5722, done.
remote: Total 5722 (delta 0), reused 0 (delta 0), pack-reused 5722
Receiving objects: 100% (5722/5722), 4.47 MiB | 14.41 MiB/s, done.
Resolving deltas: 100% (3932/3932), done.
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/gpt4all/gpt4all-backend/llama.cpp-mainline'...
remote: Enumerating objects: 3643, done.
remote: Total 3643 (delta 0), reused 0 (delta 0), pack-reused 3643
Receiving objects: 100% (3643/3643), 3.35 MiB | 7.87 MiB/s, done.
Resolving deltas: 100% (2434/2434), done.
Submodule path 'gpt4all-backend/llama.cpp-230511': checked out 'f826aac617e1c5847ecb5115f75433aff82f759a'
Submodule path 'gpt4all-backend/llama.cpp-230519': checked out '5ea43392731040b454c293123839b90e159cbb99'
Submodule path 'gpt4all-backend/llama.cpp-mainline': checked out 'da760ac3829a89ab9d60ec797df8a570b9b8419a'
cd gpt4all && git checkout -b build 8aba2c9009fb6bc723f623c614e265b41722e4e3 && git submodule update --init --recursive --depth 1
Switched to a new branch 'build'
git clone --recurse-submodules https://github.com/mudler/go-piper go-piper
Cloning into 'go-piper'...
remote: Enumerating objects: 37, done.
remote: Counting objects: 100% (37/37), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 37 (delta 16), reused 31 (delta 10), pack-reused 0
Unpacking objects: 100% (37/37), 6.19 KiB | 12.00 KiB/s, done.
Submodule 'piper' (https://github.com/rhasspy/piper) registered for path 'piper'
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-piper/piper'...
remote: Enumerating objects: 1130, done.
remote: Counting objects: 100% (317/317), done.
remote: Compressing objects: 100% (122/122), done.
remote: Total 1130 (delta 211), reused 235 (delta 194), pack-reused 813
Receiving objects: 100% (1130/1130), 66.21 MiB | 19.29 MiB/s, done.
Resolving deltas: 100% (627/627), done.
Submodule path 'piper': checked out 'eb563c1af57c1205e422d4a573eec8eb2ed1e525'
cd go-piper && git checkout -b build 56b8a81b4760a6fbee1a82e62f007ae7e8f010a7 && git submodule update --init --recursive --depth 1
Switched to a new branch 'build'
git clone --recurse-submodules https://github.com/donomii/go-rwkv.cpp go-rwkv
Cloning into 'go-rwkv'...
remote: Enumerating objects: 257, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 257 (delta 2), reused 1 (delta 0), pack-reused 242
Receiving objects: 100% (257/257), 5.17 MiB | 14.88 MiB/s, done.
Resolving deltas: 100% (129/129), done.
Submodule 'rwkv.cpp' (https://github.com/saharNooby/rwkv.cpp) registered for path 'rwkv.cpp'
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-rwkv/rwkv.cpp'...
remote: Enumerating objects: 1904, done.
remote: Counting objects: 100% (503/503), done.
remote: Compressing objects: 100% (190/190), done.
remote: Total 1904 (delta 372), reused 370 (delta 295), pack-reused 1401
Receiving objects: 100% (1904/1904), 5.70 MiB | 14.08 MiB/s, done.
Resolving deltas: 100% (1171/1171), done.
Submodule path 'rwkv.cpp': checked out '6b26e0db28b26f0fb2c73c5aa6ff490818fb1456'
Submodule 'ggml' (https://github.com/ggerganov/ggml) registered for path 'rwkv.cpp/ggml'Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-rwkv/rwkv.cpp/ggml'...
remote: Enumerating objects: 2963, done.
remote: Counting objects: 100% (962/962), done.
remote: Compressing objects: 100% (136/136), done.
remote: Total 2963 (delta 872), reused 858 (delta 814), pack-reused 2001
Receiving objects: 100% (2963/2963), 4.72 MiB | 7.26 MiB/s, done.
Resolving deltas: 100% (1955/1955), done.
Submodule path 'rwkv.cpp/ggml': checked out 'f52d2a05cf8327baf6c0d49e7b231953179e03d3'
cd go-rwkv && git checkout -b build c898cd0f62df8f2a7830e53d1d513bef4f6f792b && git submodule update --init --recursive --depth 1
Switched to a new branch 'build'
git clone https://github.com/ggerganov/whisper.cpp.git
Cloning into 'whisper.cpp'...
remote: Enumerating objects: 4086, done.
remote: Counting objects: 100% (71/71), done.
remote: Compressing objects: 100% (54/54), done.
remote: Total 4086 (delta 26), reused 44 (delta 15), pack-reused 4015
Receiving objects: 100% (4086/4086), 6.97 MiB | 8.69 MiB/s, done.
Resolving deltas: 100% (2556/2556), done.
Updating files: 100% (304/304), done.
cd whisper.cpp && git checkout -b build 85ed71aaec8e0612a84c0b67804bde75aa75a273 && git
submodule update --init --recursive --depth 1
Switched to a new branch 'build'
Submodule 'bindings/ios' (https://github.com/ggerganov/whisper.spm) registered for path
'bindings/ios'
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/whisper.cpp/bindings/ios'...
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 5 (delta 4), reused 1 (delta 0), pack-reused 0
Unpacking objects: 100% (5/5), 1.05 KiB | 19.00 KiB/s, done.
From https://github.com/ggerganov/whisper.spm
* branch de46d9e7817fe851c109d66080239d415812d32a -> FETCH_HEAD
Submodule path 'bindings/ios': checked out 'de46d9e7817fe851c109d66080239d415812d32a'
git clone --recurse-submodules https://github.com/go-skynet/go-bert.cpp go-bert
Cloning into 'go-bert'...
remote: Enumerating objects: 113, done.
remote: Counting objects: 100% (17/17), done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 113 (delta 7), reused 5 (delta 3), pack-reused 96
Receiving objects: 100% (113/113), 34.16 KiB | 2.01 MiB/s, done.
Resolving deltas: 100% (46/46), done.
Submodule 'bert.cpp' (https://github.com/skeskinen/bert.cpp) registered for path 'bert.cpp'
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-bert/bert.cpp'...
remote: Enumerating objects: 246, done.
remote: Counting objects: 100% (37/37), done.
remote: Compressing objects: 100% (28/28), done.
remote: Total 246 (delta 19), reused 23 (delta 9), pack-reused 209
Receiving objects: 100% (246/246), 171.23 KiB | 3.80 MiB/s, done.
Resolving deltas: 100% (137/137), done.
Submodule path 'bert.cpp': checked out '848366fc0718ba11d477ad56ea8a43fbcf104d9c'
Submodule 'ggml' (https://github.com/ggerganov/ggml.git) registered for path 'bert.cpp/ggml'
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-bert/bert.cpp/ggml'...
remote: Enumerating objects: 2963, done.
remote: Counting objects: 100% (962/962), done.
remote: Compressing objects: 100% (136/136), done.
remote: Total 2963 (delta 872), reused 858 (delta 814), pack-reused 2001
Receiving objects: 100% (2963/2963), 4.72 MiB | 12.42 MiB/s, done.
Resolving deltas: 100% (1955/1955), done.
Submodule path 'bert.cpp/ggml': checked out '94a24c9de987fa7a551310a8ba4286e7715d8950'
cd go-bert && git checkout -b build 6abe312cded14042f6b7c3cd8edf082713334a4d && git submodule update --init --recursive --depth 1
Switched to a new branch 'build'
git clone --recurse-submodules https://github.com/go-skynet/bloomz.cpp bloomz
Cloning into 'bloomz'...
remote: Enumerating objects: 292, done.
remote: Counting objects: 100% (34/34), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 292 (delta 22), reused 18 (delta 18), pack-reused 258
Receiving objects: 100% (292/292), 2.18 MiB | 11.27 MiB/s, done.
Resolving deltas: 100% (146/146), done.
cd bloomz && git checkout -b build 1834e77b83faafe912ad4092ccf7f77937349e2f && git submodule update --init --recursive --depth 1
Switched to a new branch 'build'
git clone --recurse-submodules https://github.com/mudler/go-stable-diffusion go-stable-diffusion
Cloning into 'go-stable-diffusion'...
remote: Enumerating objects: 75, done.
remote: Counting objects: 100% (75/75), done.
remote: Compressing objects: 100% (46/46), done.
remote: Total 75 (delta 40), reused 62 (delta 27), pack-reused 0
Unpacking objects: 100% (75/75), 5.45 MiB | 2.72 MiB/s, done.
Submodule 'ncnn' (https://github.com/Tencent/ncnn) registered for path 'ncnn'
Submodule 'stable-diffusion' (https://github.com/EdVince/Stable-Diffusion-NCNN) registered for path 'stable-diffusion'
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-stable-diffusion/ncnn'...
remote: Enumerating objects: 31746, done.
remote: Counting objects: 100% (5994/5994), done.
remote: Compressing objects: 100% (557/557), done.
remote: Total 31746 (delta 5735), reused 5471 (delta 5437), pack-reused 25752
Receiving objects: 100% (31746/31746), 22.29 MiB | 14.88 MiB/s, done.
Resolving deltas: 100% (26924/26924), done.
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-stable-diffusion/stable-diffusion'...
remote: Enumerating objects: 1412, done.
remote: Counting objects: 100% (192/192), done.
remote: Compressing objects: 100% (141/141), done.
remote: Total 1412 (delta 50), reused 134 (delta 38), pack-reused 1220
Receiving objects: 100% (1412/1412), 154.13 MiB | 13.71 MiB/s, done.
Resolving deltas: 100% (630/630), done.
Submodule path 'ncnn': checked out '2b87dc2cf7f07416c3781e71a478fd185d4fdfcb'
Submodule 'glslang' (https://github.com/KhronosGroup/glslang) registered for path 'ncnn/glslang'
Submodule 'python/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'ncnn/python/pybind11'
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-stable-diffusion/ncnn/glslang'...
remote: Enumerating objects: 53935, done.
remote: Counting objects: 100% (696/696), done.
remote: Compressing objects: 100% (375/375), done.
remote: Total 53935 (delta 419), reused 529 (delta 313), pack-reused 53239
Receiving objects: 100% (53935/53935), 69.97 MiB | 16.04 MiB/s, done.
Resolving deltas: 100% (45221/45221), done.
Cloning into '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-stable-diffusion/ncnn/python/pybind11'...
remote: Enumerating objects: 26770, done.
remote: Counting objects: 100% (689/689), done.
remote: Compressing objects: 100% (270/270), done.
remote: Total 26770 (delta 437), reused 565 (delta 375), pack-reused 26081
Receiving objects: 100% (26770/26770), 10.38 MiB | 15.15 MiB/s, done.
Resolving deltas: 100% (18757/18757), done.
Submodule path 'ncnn/glslang': checked out '88fd417b0bb7d91755961c70e846d274c182f2b0'
Submodule path 'ncnn/python/pybind11': checked out '80dc998efced8ceb2be59756668a7e90e8bef917'
Submodule path 'stable-diffusion': checked out 'bac5d4a71a442e9e9fab4a1e30ca203a7c7d3272'
cd go-stable-diffusion && git checkout -b build d89260f598afb809279bc72aa0107b4292587632 && git submodule update --init --recursive --depth 1
Switched to a new branch 'build'
touch get-sources
go mod edit -replace github.com/go-skynet/go-llama.cpp=/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-llama
go mod edit -replace github.com/go-skynet/go-llama.cpp-grammar=/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-llama-grammar
go mod edit -replace github.com/nomic-ai/gpt4all/gpt4all-bindings/golang=/mnt/d/Projects/AIProjects/LocalAI/LocalAI/gpt4all/gpt4all-bindings/golang
go mod edit -replace github.com/go-skynet/go-ggml-transformers.cpp=/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-ggml-transformers
go mod edit -replace github.com/donomii/go-rwkv.cpp=/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-rwkv
go mod edit -replace github.com/ggerganov/whisper.cpp=/mnt/d/Projects/AIProjects/LocalAI/LocalAI/whisper.cpp
go mod edit -replace github.com/go-skynet/go-bert.cpp=/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-bert
go mod edit -replace github.com/go-skynet/bloomz.cpp=/mnt/d/Projects/AIProjects/LocalAI/LocalAI/bloomz
go mod edit -replace github.com/mudler/go-stable-diffusion=/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-stable-diffusion
go mod edit -replace github.com/mudler/go-piper=/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-piper
go mod edit -replace github.com/mudler/go-ggllm.cpp=/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-ggllm
go mod download
touch prepare
mkdir -p backend-assets/grpc
go build -ldflags " -X "github.com/go-skynet/LocalAI/internal.Version=v1.22.0-2-g72e3e23" -X "github.com/go-skynet/LocalAI/internal.Commit=72e3e236de2962861558919041b45ece48cbfb34"" -tags "" -o backend-assets/grpc/langchain-huggingface ./cmd/grpc/langchain-huggingface/
make -C go-llama-grammar BUILD_TYPE=cublas libbinding.a
make[1]: Entering directory '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-llama-grammar'
I llama.cpp build info:
I UNAME_S: Linux
I UNAME_P: x86_64
I UNAME_M: x86_64
I CFLAGS: -I./llama.cpp -I. -O3 -DNDEBUG -std=c11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wno-unused-function -pthread -march=native -mtune=native
I CXXFLAGS: -I./llama.cpp -I. -I./llama.cpp/examples -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -pthread
I CGO_LDFLAGS:
I LDFLAGS:
I BUILD_TYPE: cublas
I CMAKE_ARGS: -DLLAMA_CUBLAS=ON
I EXTRA_TARGETS: llama.cpp/ggml-cuda.o
I CC: cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
I CXX: g++ (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
cd build && cp -rf CMakeFiles/ggml.dir/ggml-cuda.cu.o ../llama.cpp/ggml-cuda.o
cp: cannot stat 'CMakeFiles/ggml.dir/ggml-cuda.cu.o': No such file or directory
make[1]: *** [Makefile:184: llama.cpp/ggml-cuda.o] Error 1
make[1]: Leaving directory '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-llama-grammar'make: *** [Makefile:216: go-llama-grammar/libbinding.a] Error 2
I can second this error on Ubuntu. The error message points to a potential version mismatch between the Cuda toolkit and Nvidia drivers though.
I'm also wondering if it could be a problem with incompatibility between the machine's CUDA driver version and the CUDA runtime version inside the Docker container.
@nibiru5 did u manage to solve it?
+1 for this error. I'm in Ubuntu on WSL2, running inside a container.
@rlleshi how would I confirm/deny if I have an incompatibility between my machine's CUDA driver version and the CUDA runtime version inside the container?
It seems CMake's auto-detection of the CUDA libraries picks up the native Windows executables/libraries and tries to use them, which obviously fails. You need to explicitly tell it to use the Linux cudatoolkit:
make BUILD_TYPE=cublas CUDAToolkit_ROOT=/usr/local/cuda CUDA_PATH=/usr/local/cuda build
It seems CMake's auto-detection of the CUDA libraries picks up the native Windows executables/libraries and tries to use them, which obviously fails. You need to explicitly tell it to use the Linux cudatoolkit:
make BUILD_TYPE=cublas CUDAToolkit_ROOT=/usr/local/cuda CUDA_PATH=/usr/local/cuda build
I tried that and got a slightly different error:
g++ -I. -I./ggml.cpp/include -I./ggml.cpp/include/ggml/ -I./ggml.cpp/examples/ -O3 -DNDEBUG -std=c++17 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -c ggml.cpp/examples/common.cpp -o common.o
ar src libtransformers.a replit.o gptj.o mpt.o gptneox.o starcoder.o gpt2.o dolly.o falcon.o ggml.o common-ggml.o common.o ggml-cuda.o
ar: ggml-cuda.o: No such file or directory
After installing instructions on https://docs.nvidia.com/cuda/wsl-user-guide/index.html which included (for my setup) installing: https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local I got past the missing ggml-cuda.o error.
Running in a wsl window, I see I am missing zlib. And openssl. and ... so it goes...
sudo apt install build-essential checkinstall zlib1g-dev -y
sudo apt install openssl
sudo apt install -y protobuf-compiler
sudo apt -y install protobuf-compiler-grpc
Now I am getting:
[ 92%] Building CXX object examples/grpc-server/CMakeFiles/hw_grpc_proto.dir/backend.grpc.pb.cc.o
In file included from /mnt/c/Users/danie/LocalAI/backend/cpp/llama/llama.cpp/build/examples/grpc-server/backend.grpc.pb.cc:6:
/mnt/c/Users/danie/LocalAI/backend/cpp/llama/llama.cpp/build/examples/grpc-server/backend.grpc.pb.h:10:10: fatal error:grpc/impl/codegen/port_platform.h: No such file or directory
10 | #include <grpc/impl/codegen/port_platform.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Also I needed:
sudo apt install -y libgrpc-dev
sudo apt install -y libgrpc++-dev
But now I am getting a ton of errors involving protobuf. The first error is:
[ 95%] Building CXX object examples/grpc-server/CMakeFiles/grpc-server.dir/grpc-server.cpp.o
In file included from /mnt/c/Users/danie/LocalAI/backend/cpp/llama/llama.cpp/examples/grpc-server/grpc-server.cpp:20:
/mnt/c/Users/danie/LocalAI/backend/cpp/llama/llama.cpp/build/examples/grpc-server/backend.pb.h:17:2: error: #error This file was generated by an older version of protoc which is
17 | #error This file was generated by an older version of protoc which is
| ^~~~~
/mnt/c/Users/danie/LocalAI/backend/cpp/llama/llama.cpp/build/examples/grpc-server/backend.pb.h:18:2: error: #error incompatible with your Protocol Buffer headers. Please
18 | #error incompatible with your Protocol Buffer headers. Please
| ^~~~~
/mnt/c/Users/danie/LocalAI/backend/cpp/llama/llama.cpp/build/examples/grpc-server/backend.pb.h:19:2: error: #error regenerate this file with a newer version of protoc.
19 | #error regenerate this file with a newer version of protoc.
| ^~~~~
Don't think I can fix that by installing more libs. I think maybe since these examples seem broken, I'll remove them or disable attempting to compile them.
I was able to disable compilation and inclusion of the broken code. Maybe the most important change was to remove backend-assets/grpc/llama-cpp from the ALL_GRPC_BACKENDS variable in the top-level Makefile:
# Removed backend-assets/grpc/llama-cpp
ALL_GRPC_BACKENDS=backend-assets/grpc/langchain-huggingface backend-assets/grpc/falcon-ggml backend-assets/grpc/bert-embeddings backend-assets/grpc/llama backend-assets/grpc/llama-ggml backend-assets/grpc/gpt4all backend-assets/grpc/dolly backend-assets/grpc/gpt2 backend-assets/grpc/gptj backend-assets/grpc/gptneox backend-assets/grpc/mpt backend-assets/grpc/replit backend-assets/grpc/starcoder backend-assets/grpc/rwkv backend-assets/grpc/whisper $(OPTIONAL_GRPC)
And then, after a few more minutes of compiling, it made it to the end, producing a working local-ai binary. No idea what I might have broken in the process. Now to test it...
i got everything fixeed "solution" it was just i completed removed that ubuntu WSL and got a new Debian WSL which is very much better than it !! i just download the release "local-ai-avx-Linux-x86_64" and ran it with a .sh script.
localai.sh
#!/bin/sh
export GALLERIES='[{"url": "github:go-skynet/model-gallery/huggingface.yaml","name":"huggingface"}]'
export CORS=true
export ADDRESS=localhost:4040
export DEBUG=true
export PARALLEL_REQUESTS=true
# export WATCHDOG_IDLE=true
export THREADS=18
./local-ai-avx-Linux-x86_64
i am using the model : thebloke__wizardlm-13b-v1-0-uncensored-superhot-8k-ggml__wizardlm-13b-v1.0-superhot-8k.ggmlv3.q4_k_m.bin
, only tested with this model for now.
It works like charm 🪄 🌟
LocalAI version: 1.22.0
Environment, CPU architecture, OS, and Version: WSL Ubuntu via VSCode Intel x86 i5-10400 Nvidia GTX 1070 Windows 10 21H1
uname -a output: Linux DESKTOP-CU0RN3K 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Describe the bug When building on WSL ubuntu, I see this error ... cd build && cp -rf CMakeFiles/ggml.dir/k_quants.c.o ../llama.cpp/k_quants.o cd build && cp -rf CMakeFiles/ggml.dir/ggml-cuda.cu.o ../llama.cpp/ggml-cuda.o cp: cannot stat 'CMakeFiles/ggml.dir/ggml-cuda.cu.o': No such file or directory make[1]: [Makefile:184: llama.cpp/ggml-cuda.o] Error 1 make[1]: Leaving directory '/mnt/d/Projects/AIProjects/LocalAI/LocalAI/go-llama-grammar'make: [Makefile:216: go-llama-grammar/libbinding.a] Error 2
To Reproduce When running this: make BUILD_TYPE=cublas build
It goes perfectly fine until it results in the above error I have attempted make clean, make rebuild, and I unstalled cmake 3.27.1 and golang 1.18.3
Expected behavior It should build successfully without error
Logs Can provide build output if needed...
Additional context I'm less experienced with building on WSL so open to feedback Thank you