Open jamesbraza opened 11 months ago
Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!
_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.
Don't engage in conversation with me, I don't support (yet) replying!
ERROR: The prompt size exceeds the context window size and cannot be processed.
Sources:
I got the same issue, running with "colima" on MacOS with k3d + containerd.
=> ERROR [linux/arm64 requirements-extras 1/4] RUN curl https://repo.anaconda.com/pkgs/misc/gpgkeys/anaconda.asc | gpg 3.7s
------
> [linux/arm64 requirements-extras 1/4] RUN curl https://repo.anaconda.com/pkgs/misc/gpgkeys/anaconda.asc | gpg --dearmor > conda.gpg && install -o root -g root -m 644 conda.gpg /usr/share/keyrings/conda-archive-keyring.gpg && gpg --keyring /usr/share/keyrings/conda-archive-keyring.gpg --no-default-keyring --fingerprint 34161F5BF5EB1D4BFBBB8F0A8AEB4F8B29D82806 && echo "deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main" > /etc/apt/sources.list.d/conda.list && echo "deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main" | tee -a /etc/apt/sources.list.d/conda.list && apt-get update && apt-get install -y conda && apt-get clean:
0.085 % Total % Received % Xferd Average Speed Time Time Time Current
0.085 Dload Upload Total Spent Left Speed
100 1709 100 1709 0 0 4049 0 --:--:-- --:--:-- --:--:-- 4049
0.511 gpg: directory '/root/.gnupg' created
0.512 gpg: /root/.gnupg/trustdb.gpg: trustdb created
0.512 pub rsa2048 2018-11-02 [SC]
0.512 3416 1F5B F5EB 1D4B FBBB 8F0A 8AEB 4F8B 29D8 2806
0.512 uid [ unknown] Distribution Team <distribution_team@anaconda.com>
0.512 sub rsa2048 2018-11-02 [E]
0.512
0.513 deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main
2.529 Hit:1 http://deb.debian.org/debian bullseye InRelease
2.529 Hit:2 http://deb.debian.org/debian-security bullseye-security InRelease
2.535 Hit:3 http://deb.debian.org/debian bullseye-updates InRelease
2.889 Get:4 https://repo.anaconda.com/pkgs/misc/debrepo/conda stable InRelease [3960 B]
3.016 Get:5 https://repo.anaconda.com/pkgs/misc/debrepo/conda stable/main amd64 Packages [3857 B]
3.024 Fetched 7817 B in 2s (3132 B/s)
3.024 Reading package lists...
3.315 W: Target Packages (main/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/conda.list:1 and /etc/apt/sources.list.d/conda.list:2
3.315 W: Target Packages (main/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/conda.list:1 and /etc/apt/sources.list.d/conda.list:2
3.315 W: Target Packages (main/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/conda.list:1 and /etc/apt/sources.list.d/conda.list:2
3.315 W: Target Packages (main/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/conda.list:1 and /etc/apt/sources.list.d/conda.list:2
3.321 Reading package lists...
3.577 Building dependency tree...
3.641 Reading state information...
3.683 W: Target Packages (main/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/conda.list:1 and /etc/apt/sources.list.d/conda.list:2
3.683 W: Target Packages (main/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/conda.list:1 and /etc/apt/sources.list.d/conda.list:2
3.683 E: Unable to locate package conda
And thinking this could be the reason: https://stackoverflow.com/questions/76879889/conda-package-not-found-how-to-install-conda-packages-on-apple-m1-m2-chips-whi
But according to anaconda's official site, it should already have ARM64 builds, so the reason should be the missing ARM64 builds in the APT repo? https://docs.anaconda.com/free/anaconda/install/linux/
Disclaimer: I'm a random person, that spent 10 min. looking at this project.
Seems like conda package is only available for amd64:
o# apt search conda
Sorting... Done
Full Text Search... Done
conda/stable 23.11.0-0 amd64
OS-agnostic, system-level binary package manager and ecosystem
I would recommend replacing https://github.com/mudler/LocalAI/blob/master/Dockerfile#L60-L66 with something like:
RUN curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-$(uname -m).sh -o ~/miniconda.sh && \
bash ~/miniconda.sh -b -p $HOME/miniconda && \
eval "$(~/miniconda/bin/conda shell.bash hook)" && \
conda init
Also, I'm not sure if you get the benefits from metal framework with Linux kernel.
I just went to anaconda.com, downloaded, installed, removed https://github.com/mudler/LocalAI/blob/master/Dockerfile#L60-L66 altogether and it went through. The build takes forever though.
Until (TBC)
=> ERROR [stage-3 10/23] RUN if [ "extras" = "extras" ]; then PATH=/root/.cargo/bin:/usr/local/cuda/bin:/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt 0.3s
@i11 I tried it with your suggestion, but then I got the following error:
failed to solve: process "/bin/sh -c if [ \"${IMAGE_TYPE}\" = \"extras\" ]; then \tPATH=$PATH:/opt/conda/bin make -C backend/python/autogptq ; fi" did not complete successfully: exit code: 2
services:
api:
...
build:
...
+ platforms:
+ - "linux/amd64"
This should help
services: api: ... build: ... + platforms: + - "linux/amd64"
This should help
With this it goes further, but then the following error is thrown:
19.17 -- Configuring incomplete, errors occurred!
19.17 See also "/build/sources/go-ggml-transformers/build/CMakeFiles/CMakeOutput.log".
19.17 See also "/build/sources/go-ggml-transformers/build/CMakeFiles/CMakeError.log".
19.18 make[1]: *** [Makefile:150: ggml.o] Error 1
19.18 make[1]: Leaving directory '/build/sources/go-ggml-transformers'
19.18 make: *** [Makefile:226: sources/go-ggml-transformers/libtransformers.a] Error 2
------
failed to solve: process "/bin/sh -c make build" did not complete successfully: exit code: 2
`docker-compose` process finished with exit code 17
seems to be an error while building the "go-ggml-transformers" dependency; more specifically, during the CMake configuration process in the tests/CMakeLists.txt file
services: api: ... build: ... + platforms: + - "linux/amd64"
This should help
With this it goes further, but then the following error is thrown:
19.17 -- Configuring incomplete, errors occurred! 19.17 See also "/build/sources/go-ggml-transformers/build/CMakeFiles/CMakeOutput.log". 19.17 See also "/build/sources/go-ggml-transformers/build/CMakeFiles/CMakeError.log". 19.18 make[1]: *** [Makefile:150: ggml.o] Error 1 19.18 make[1]: Leaving directory '/build/sources/go-ggml-transformers' 19.18 make: *** [Makefile:226: sources/go-ggml-transformers/libtransformers.a] Error 2 ------ failed to solve: process "/bin/sh -c make build" did not complete successfully: exit code: 2 `docker-compose` process finished with exit code 17
seems to be an error while building the "go-ggml-transformers" dependency; more specifically, during the CMake configuration process in the tests/CMakeLists.txt file
That should be fixed once #1689 is merged
Conda is still blocking me from completing the build, even after modifying my Dockerfile per @i11 recommendation. Any tips?
Here is my docker-compose file:
version: '3.6'
services:
api:
image: quay.io/go-skynet/local-ai:latest
build:
context: .
dockerfile: Dockerfile
platforms:
- "linux/arm64"
ports:
- 8080:8080
env_file:
- .env
volumes:
- ./models:/models:cached
- ./images/:/tmp/generated/images/
command: ["/usr/bin/local-ai" ]
runtime: nvidia
And here is my modified DockerFile per @i11 quick recommendation (which did get me past the original Unable to locate package conda
error:
ARG IMAGE_TYPE=extras
ARG BASE_IMAGE=ubuntu:22.04
# extras or core
FROM ${BASE_IMAGE} as requirements-core
ARG GO_VERSION=1.21.7
ARG BUILD_TYPE
ARG CUDA_MAJOR_VERSION=11
ARG CUDA_MINOR_VERSION=7
ARG TARGETARCH
ARG TARGETVARIANT
ENV BUILD_TYPE=${BUILD_TYPE}
ENV DEBIAN_FRONTEND=noninteractive
ENV EXTERNAL_GRPC_BACKENDS="coqui:/build/backend/python/coqui/run.sh,huggingface-embeddings:/build/backend/python/sentencetransformers/run.sh,petals:/build/backend/python/petals/run.sh,transformers:/build/backend/python/transformers/run.sh,sentencetransformers:/build/backend/python/sentencetransformers/run.sh,autogptq:/build/backend/python/autogptq/run.sh,bark:/build/backend/python/bark/run.sh,diffusers:/build/backend/python/diffusers/run.sh,exllama:/build/backend/python/exllama/run.sh,vall-e-x:/build/backend/python/vall-e-x/run.sh,vllm:/build/backend/python/vllm/run.sh,mamba:/build/backend/python/mamba/run.sh,exllama2:/build/backend/python/exllama2/run.sh,transformers-musicgen:/build/backend/python/transformers-musicgen/run.sh"
ARG GO_TAGS="stablediffusion tinydream tts"
RUN apt-get update && \
apt-get install -y ca-certificates curl patch pip cmake git && apt-get clean
# Install Go
RUN curl -L -s https://go.dev/dl/go$GO_VERSION.linux-$TARGETARCH.tar.gz | tar -v -C /usr/local -xz
ENV PATH $PATH:/usr/local/go/bin
COPY --chmod=644 custom-ca-certs/* /usr/local/share/ca-certificates/
RUN update-ca-certificates
# Use the variables in subsequent instructions
RUN echo "Target Architecture: $TARGETARCH"
RUN echo "Target Variant: $TARGETVARIANT"
# CuBLAS requirements
RUN if [ "${BUILD_TYPE}" = "cublas" ]; then \
apt-get install -y software-properties-common && \
curl -O https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb && \
dpkg -i cuda-keyring_1.1-1_all.deb && \
rm -f cuda-keyring_1.1-1_all.deb && \
apt-get update && \
apt-get install -y cuda-nvcc-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} libcurand-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} libcublas-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} libcusparse-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} libcusolver-dev-${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION} && apt-get clean \
; fi
# Cuda
ENV PATH /usr/local/cuda/bin:${PATH}
# HipBLAS requirements
ENV PATH /opt/rocm/bin:${PATH}
# OpenBLAS requirements and stable diffusion
RUN apt-get install -y \
libopenblas-dev \
libopencv-dev \
&& apt-get clean
# Set up OpenCV
RUN ln -s /usr/include/opencv4/opencv2 /usr/include/opencv2
WORKDIR /build
RUN test -n "$TARGETARCH" \
|| (echo 'warn: missing $TARGETARCH, either set this `ARG` manually, or run using `docker buildkit`')
# Extras requirements
FROM requirements-core as requirements-extras
**#RUN curl https://repo.anaconda.com/pkgs/misc/gpgkeys/anaconda.asc | gpg --dearmor > conda.gpg && \
# install -o root -g root -m 644 conda.gpg /usr/share/keyrings/conda-archive-keyring.gpg && \
# gpg --keyring /usr/share/keyrings/conda-archive-keyring.gpg --no-default-keyring --fingerprint 34161F5BF5EB1D4BFBBB8F0A8AEB4F8B29D82806 && \
# echo "deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main" > /etc/apt/sources.list.d/conda.list && \
# echo "deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main" | tee -a /etc/apt/sources.list.d/conda.list && \
# apt-get update && \
# apt-get install -y conda && apt-get clean
RUN curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-$(uname -m).sh -o ~/miniconda.sh && \
bash ~/miniconda.sh -b -p $HOME/miniconda && \
eval "$(~/miniconda/bin/conda shell.bash hook)" && \
conda init**
ENV PATH="/root/.cargo/bin:${PATH}"
RUN apt-get install -y python3-pip && apt-get clean
RUN pip install --upgrade pip
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
RUN apt-get install -y espeak-ng espeak && apt-get clean
###################################
###################################
FROM requirements-${IMAGE_TYPE} as builder
ARG GO_TAGS="stablediffusion tts"
ARG GRPC_BACKENDS
ARG BUILD_GRPC=true
ENV GRPC_BACKENDS=${GRPC_BACKENDS}
ENV GO_TAGS=${GO_TAGS}
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
ENV NVIDIA_REQUIRE_CUDA="cuda>=${CUDA_MAJOR_VERSION}.0"
ENV NVIDIA_VISIBLE_DEVICES=all
WORKDIR /build
COPY . .
COPY .git .
RUN make prepare
# stablediffusion does not tolerate a newer version of abseil, build it first
RUN GRPC_BACKENDS=backend-assets/grpc/stablediffusion make build
RUN if [ "${BUILD_GRPC}" = "true" ]; then \
git clone --recurse-submodules -b v1.58.0 --depth 1 --shallow-submodules https://github.com/grpc/grpc && \
cd grpc && mkdir -p cmake/build && cd cmake/build && cmake -DgRPC_INSTALL=ON \
-DgRPC_BUILD_TESTS=OFF \
../.. && make -j12 install \
; fi
# Rebuild with defaults backends
RUN make build
RUN if [ ! -d "/build/sources/go-piper/piper-phonemize/pi/lib/" ]; then \
mkdir -p /build/sources/go-piper/piper-phonemize/pi/lib/ \
touch /build/sources/go-piper/piper-phonemize/pi/lib/keep \
; fi
###################################
###################################
FROM requirements-${IMAGE_TYPE}
ARG FFMPEG
ARG BUILD_TYPE
ARG TARGETARCH
ARG IMAGE_TYPE=extras
ENV BUILD_TYPE=${BUILD_TYPE}
ENV REBUILD=false
ENV HEALTHCHECK_ENDPOINT=http://localhost:8080/readyz
ARG CUDA_MAJOR_VERSION=11
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
ENV NVIDIA_REQUIRE_CUDA="cuda>=${CUDA_MAJOR_VERSION}.0"
ENV NVIDIA_VISIBLE_DEVICES=all
ENV PIP_CACHE_PURGE=true
# Add FFmpeg
RUN if [ "${FFMPEG}" = "true" ]; then \
apt-get install -y ffmpeg && apt-get clean \
; fi
WORKDIR /build
# we start fresh & re-copy all assets because `make build` does not clean up nicely after itself
# so when `entrypoint.sh` runs `make build` again (which it does by default), the build would fail
# see https://github.com/go-skynet/LocalAI/pull/658#discussion_r1241971626 and
# https://github.com/go-skynet/LocalAI/pull/434
COPY . .
COPY --from=builder /build/sources ./sources/
COPY --from=builder /build/grpc ./grpc/
RUN make prepare-sources && cd /build/grpc/cmake/build && make install && rm -rf grpc
# Copy the binary
COPY --from=builder /build/local-ai ./
# Copy shared libraries for piper
COPY --from=builder /build/sources/go-piper/piper-phonemize/pi/lib/* /usr/lib/
# do not let stablediffusion rebuild (requires an older version of absl)
COPY --from=builder /build/backend-assets/grpc/stablediffusion ./backend-assets/grpc/stablediffusion
## Duplicated from Makefile to avoid having a big layer that's hard to push
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/autogptq \
; fi
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/bark \
; fi
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/diffusers \
; fi
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/vllm \
; fi
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/mamba \
; fi
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/sentencetransformers \
; fi
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/transformers \
; fi
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/vall-e-x \
; fi
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/exllama \
; fi
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/exllama2 \
; fi
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/petals \
; fi
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/transformers-musicgen \
; fi
RUN if [ "${IMAGE_TYPE}" = "extras" ]; then \
PATH=$PATH:/opt/conda/bin make -C backend/python/coqui \
; fi
# Make sure the models directory exists
RUN mkdir -p /build/models
# Define the health check command
HEALTHCHECK --interval=1m --timeout=10m --retries=10 \
CMD curl -f $HEALTHCHECK_ENDPOINT || exit 1
EXPOSE 8080
ENTRYPOINT [ "/build/entrypoint.sh" ]
LocalAI version:
https://github.com/mudler/LocalAI/tree/20d637e7b70cf0e15e6bf255ab2e4c080ddde2b0 (some commits above https://github.com/mudler/LocalAI/tree/v1.40.0)
Environment, CPU architecture, OS, and Version:
I am on a MacBook Pro with macOS Ventura 13.5.2:
Describe the bug
To Reproduce
Run
docker compose up
Expected behavior
I expect
docker compose up
to workLogs
See above
Additional context
docker compose up
works as of https://github.com/mudler/LocalAI/tree/v1.40.0