mudler / LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference
https://localai.io
MIT License
25.84k stars 1.94k forks source link

patch unexpectedly ends in middle of line #3957

Open TwinFinz opened 3 weeks ago

TwinFinz commented 3 weeks ago

LocalAI version: localai/localai:latest-gpu-nvidia-cuda-12 : SHA ff0b3e63d517 (Also occurs on v2.22.1 container image)

Environment, CPU architecture, OS, and Version: Linux server 6.8.0-47-generic #47-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 27 21:40:26 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux OS Version: Ubuntu 24.04 Portainer: 2.19.5-CE CPU: intel 13900K Ram: 32gb GPU: 4090

Describe the bug (Suspected issue: "Bugged" LLAMA-CPP builds on later versions) "builds everything for about 2 hours and ends with this" Upon reproducing the "bug" it seems to be expecting input from the user.

patching file examples/llava/clip.cpp
patch unexpectedly ends in middle of line
Reversed (or previously applied) patch detected!  Assume -R? [n]

To Reproduce deploy docker using the following docker-compose.yaml

version: "3.9"
services:
  api:
    image: localai/localai:latest-gpu-nvidia-cuda-12
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/readyz"]
      interval: 1m
      timeout: 20m
      retries: 5
    restart: always
    tty: true
    ports:
      - 8080:8080
    environment:
      - TZ=America/New_York
      - NVIDIA_VISIBLE_DEVICES=0
##      - NVIDIA_DRIVER_CAPABILITIES: all
      - DEBUG=true
      - LOCALAI_WATCHDOG_IDLE=true
      - LOCAlAI_WATCHDOG_IDLE_TIMEOUT=30m
      - LOCALAI_WATCHDOG_BUSY=true
      - LOCALAI_WATCHDOG_BUSY_TIMEOUT=30m
      - BUILD_TYPE=cublas
      - LOCALAI_CORS=true
      - LOCALAI_CORS_ALLOW_ORIGINS=*
      - GO_TAGS=stablediffusion
      - REBUILD=true
      - LOCALAI_LOG_LEVEL=debug
      - LOCALAI_IMAGE_PATH=/tmp/generated/images
      - LOCALAI_AUDIO_PATH=/tmp/generated/audio
    volumes:
      - ./models:/build/models:cache
      - ./images/:/tmp/generated/images/
      - ./audio/:/tmp/generated/audio/
    runtime: nvidia
    deploy:
       resources:
         reservations:
           devices:
             - driver: nvidia
##               count: 1
               capabilities: [gpu]
               device_ids: ['0']

Expected behavior Build and launch of server

Logs Log Provided by user:

I llama-cpp build info:avx
cp -rf backend/cpp/llama backend/cpp/llama-avx
make -C backend/cpp/llama-avx purge
make[1]: Entering directory '/build/backend/cpp/llama-avx'
rm -rf llama.cpp/build
rm -rf llama.cpp/examples/grpc-server
rm -rf grpc-server
make[1]: Leaving directory '/build/backend/cpp/llama-avx'
CMAKE_ARGS=" -DGGML_AVX=on -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off" make VARIANT="llama-avx" build-llama-cpp-grpc-server
make[1]: Entering directory '/build'
echo "BUILD_GRPC_FOR_BACKEND_LLAMA is not defined."
BUILD_GRPC_FOR_BACKEND_LLAMA is not defined.
LLAMA_VERSION=96776405a17034dcfd53d3ddf5d142d34bdbb657 make -C backend/cpp/llama-avx grpc-server
make[2]: Entering directory '/build/backend/cpp/llama-avx'
mkdir -p llama.cpp/examples/grpc-server
bash prepare.sh
Applying patch 01-llava.patch
patching file examples/llava/clip.cpp
patch unexpectedly ends in middle of line

Log from v2.22.1 container:

make[2]: Leaving directory '/build/backend/cpp/grpc'
_PROTOBUF_PROTOC=/build/backend/cpp/grpc/installed_packages/bin/proto \
_GRPC_CPP_PLUGIN_EXECUTABLE=/build/backend/cpp/grpc/installed_packages/bin/grpc_cpp_plugin \
PATH="/build/backend/cpp/grpc/installed_packages/bin:/root/.cargo/bin:/opt/rocm/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/go/bin:/usr/local/go/bin" \
CMAKE_ARGS="-DLLAMA_F16C=ON -DLLAMA_AVX=ON -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_FMA=OFF  -Dcpu=x86_64+f16c -DGGML_AVX=on -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -Dabsl_DIR=/build/backend/cpp/grpc/installed_packages/lib/cmake/absl -DProtobuf_DIR=/build/backend/cpp/grpc/installed_packages/lib/cmake/protobuf -Dutf8_range_DIR=/build/backend/cpp/grpc/installed_packages/lib/cmake/utf8_range -DgRPC_DIR=/build/backend/cpp/grpc/installed_packages/lib/cmake/grpc -DCMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES=/build/backend/cpp/grpc/installed_packages/include" \
LLAMA_VERSION=45f097645efb11b6d09a5b4adbbfd7c312ac0126 \
make -C backend/cpp/llama-avx grpc-server
make[2]: Entering directory '/build/backend/cpp/llama-avx'
mkdir -p llama.cpp/examples/grpc-server
bash prepare.sh
Applying patch 01-llava.patch
patching file examples/llava/clip.cpp
patch unexpectedly ends in middle of line
Reversed (or previously applied) patch detected!  Assume -R? [n] n

Additional context I am not the one who personally experienced this issue, It is an issue reported in the #Help discord channel (Creating issue as requested) Altho i have reproduced the issue on another image.

TwinFinz commented 2 days ago

Container V2.23.0 contains same issue (assuming a llama-cpp issue)

LocalAi-GPT  | make[2]: Entering directory '/build/backend/cpp/llama-avx'
LocalAi-GPT  | mkdir -p llama.cpp/examples/grpc-server
LocalAi-GPT  | bash prepare.sh
LocalAi-GPT  | Applying patch 01-llava.patch
LocalAi-GPT  | patching file examples/llava/clip.cpp
LocalAi-GPT  | patch unexpectedly ends in middle of line