Building an image get error

[olariuromeo]

LocalAI version:

master I clone yesterday Environment, CPU architecture, OS, and Version:

Host server ubuntu 22.04, Linux office 6.5.0-17-generic #17~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Jan 16 14:32:32 UTC 2 x86_64 x86_64 x86_64 GNU/Linux go , golang 1.21.7, python 3.11.5, Docker version 24.0.7, video card nvidia 3080 10gb vram, system memory 64gb ddr4. cuda 12.2

Describe the bug

#26 1.459 go: github.com/go-skynet/go-llama.cpp@v0.0.0-20231009155254-aeba71ee8428 (replaced by /home/user/apps/Localai/sources/go-llama-ggml): reading /home/user/apps/Localai/sources/go-llama-ggml/go.mod: open /home/user/apps/Localai/sources/go-llama-ggml/go.mod: no such file or directory
#26 1.461 make: *** [Makefile:269: prepare-sources] Error 1
#26 ERROR: process "/bin/sh -c make prepare" did not complete successfully: exit code: 2

#22 [stage-3  1/23] RUN if [ "true" = "true" ]; then     apt-get install -y ffmpeg && apt-get clean     ; fi
#22 25.78 Preparing to unpack .../21-libiec61883-0_1.2.0-4build3_amd64.deb ...
#22 25.82 Unpacking libiec61883-0:amd64 (1.2.0-4build3) ...
#22 CANCELED
------
 > [builder 4/8] RUN make prepare:
0.422 go mod edit -replace github.com/donomii/go-rwkv.cpp=/build/sources/go-rwkv
0.430 go mod edit -replace github.com/ggerganov/whisper.cpp=/build/sources/whisper.cpp
0.432 go mod edit -replace github.com/ggerganov/whisper.cpp/bindings/go=/build/sources/whisper.cpp/bindings/go
0.434 go mod edit -replace github.com/go-skynet/go-bert.cpp=/build/sources/go-bert
0.437 go mod edit -replace github.com/mudler/go-stable-diffusion=/build/sources/go-stable-diffusion
0.439 go mod edit -replace github.com/M0Rf30/go-tiny-dream=/build/sources/go-tiny-dream
0.441 go mod edit -replace github.com/mudler/go-piper=/build/sources/go-piper
0.443 go mod download
1.459 go: github.com/go-skynet/go-llama.cpp@v0.0.0-20231009155254-aeba71ee8428 (replaced by /home/user/apps/Localai/sources/go-llama-ggml): reading /home/user/apps/Localai/sources/go-llama-ggml/go.mod: open /home/user/apps/Localai/sources/go-llama-ggml/go.mod: no such file or directory
1.461 make: *** [Makefile:269: prepare-sources] Error 1
------
Dockerfile:102
--------------------
 100 |     COPY . .
 101 |     COPY .git .
 102 | >>> RUN make prepare
 103 |     
 104 |     # stablediffusion does not tolerate a newer version of abseil, build it first
--------------------
ERROR: failed to solve: process "/bin/sh -c make prepare" did not complete successfully: exit code: 2

the file exist :

cat /home/user/apps/Localai/sources/go-llama-ggml/go.mod
module github.com/go-skynet/go-llama.cpp

go 1.19

require (
    github.com/onsi/ginkgo/v2 v2.11.0
    github.com/onsi/gomega v1.27.10
)

require (
    github.com/go-logr/logr v1.2.4 // indirect
    github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect
    github.com/google/go-cmp v0.5.9 // indirect
    github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38 // indirect
    golang.org/x/net v0.12.0 // indirect
    golang.org/x/sys v0.10.0 // indirect
    golang.org/x/text v0.11.0 // indirect
    golang.org/x/tools v0.9.3 // indirect
    gopkg.in/yaml.v3 v3.0.1 // indirect
)

To Reproduce

make GO_TAGS=stablediffusion,tts BUILD_TYPE=cuBLAS BUILD_GRPC_FOR_BACKEND_LLAMA=true build

docker buildx build \
  --build-arg BUILD_TYPE=cublas \
  --build-arg TARGETARCH=amd64 \
  --build-arg IMAGE_TYPE=extras \
  --build-arg GO_TAGS="stablediffusion tts" \
  --build-arg BUILD_GRPC=true \
  --build-arg FFMPEG=true \
  -t localai:v2.8.2-cublas-cuda12-ffmpeg \
  .

Expected behavior

create an image Logs

already provided Additional context

The inference without a video card works decently, I would like to see how nvidia acceleration works. For build a new image it is necessary to delete replace directive from go.mod who has this error.

[TwinFinz]

the build can sometimes fail randomly if you hit the github rate limiter. it does not throw an error until much later. I would say try again and watch the pulling of the github files and check for anything that fails while pulling. i have all of these settings the same altho i do not "make" i rebuild the image for my cpu arch. altho one thing i notice is your image is out of date. the 2.9.0 image is available. i would honestly say if you use the latest image from "quay" it will verify that it does indeed build (even master-cublas-cuda12-ffmpeg) and then you can rebuild if needed but if your cpu supports avx2 i do not think you need to rebuild.

GO_TAGS=stablediffusion,tts BUILD_TYPE=cuBLAS BUILD_GRPC_FOR_BACKEND_LLAMA=true

[olariuromeo]

the build can sometimes fail randomly if you hit the github rate limiter. it does not throw an error until much later. I would say try again and watch the pulling of the github files and check for anything that fails while pulling. i have all of these settings the same altho i do not "make" i rebuild the image for my cpu arch. altho one thing i notice is your image is out of date. the 2.9.0 image is available. i would honestly say if you use the latest image from "quay" it will verify that it does indeed build (even master-cublas-cuda12-ffmpeg) and then you can rebuild if needed but if your cpu supports avx2 i do not think you need to rebuild.

GO_TAGS=stablediffusion,tts BUILD_TYPE=cuBLAS BUILD_GRPC_FOR_BACKEND_LLAMA=true

You're right, it's not mandatory to recreate the image, you can use the default one to make it work, but I want to make some changes and if I can't build an image with cuda support, I can't do them, and my resources are quite limited in the system and I would I want to make a lighter image just to work for my server. I really don't understand why I can't create a custom image for my system. . My cpu supports avx2 and I have 64 gb of ram, but the video card only has 10 gb of vram And I would like to experiment with making some images each with different characteristics and some small change, one with stablediffusion, one with ttts support and another without stablediffusion and tts support with cuda12 but none of them work properly. I can build the image without error only if I delete the replace directive from go mod, I tried with different versions of golan and python but when I build the container, and start the inference, with not cuda support although it shows me that the image has grpc backend with cublas, so it is useless, and the existing documentation is quite lacking

[olariuromeo]

the build can sometimes fail randomly if you hit the github rate limiter. it does not throw an error until much later. I would say try again and watch the pulling of the github files and check for anything that fails while pulling. i have all of these settings the same altho i do not "make" i rebuild the image for my cpu arch. altho one thing i notice is your image is out of date. the 2.9.0 image is available. i would honestly say if you use the latest image from "quay" it will verify that it does indeed build (even master-cublas-cuda12-ffmpeg) and then you can rebuild if needed but if your cpu supports avx2 i do not think you need to rebuild.

GO_TAGS=stablediffusion,tts BUILD_TYPE=cuBLAS BUILD_GRPC_FOR_BACKEND_LLAMA=true

tts: enables text-to-speech with go-piper requires REBUILD=true

[cihatgenc]

Have exactly the same issue on a macos. Don't think this has to do anything with github rate limiting as the file already exists locally.

[olariuromeo]

the build can sometimes fail randomly if you hit the github rate limiter. it does not throw an error until much later. I would say try again and watch the pulling of the github files and check for anything that fails while pulling. i have all of these settings the same altho i do not "make" i rebuild the image for my cpu arch. altho one thing i notice is your image is out of date. the 2.9.0 image is available. i would honestly say if you use the latest image from "quay" it will verify that it does indeed build (even master-cublas-cuda12-ffmpeg) and then you can rebuild if needed but if your cpu supports avx2 i do not think you need to rebuild.

GO_TAGS=stablediffusion,tts BUILD_TYPE=cuBLAS BUILD_GRPC_FOR_BACKEND_LLAMA=true "your image is out of date" when you build a local image, it really doesn't matter what name you give to the image.