search

mistralai / mistral-inference

Official inference library for Mistral models
https://mistral.ai/
Apache License 2.0
9.37k stars 817 forks source link

vLLM Build Issue using the provided Dockerfile #99

Closed Good-Coffee closed 7 months ago

Good-Coffee commented 7 months ago

Hi, the provided Dockerfile yields the following output. I added the whole error log.

docker build deploy --progress=plain --build-arg MAX_JOBS=18

0 building with "desktop-linux" instance using docker driver

1 [internal] load .dockerignore

1 transferring context: 57B done

1 DONE 0.0s

2 [internal] load build definition from Dockerfile

2 transferring dockerfile: 1.15kB done

2 DONE 0.0s

3 [internal] load metadata for nvcr.io/nvidia/cuda:12.1.0-devel-ubuntu22.04

3 DONE 2.1s

4 [ 1/10] FROM nvcr.io/nvidia/cuda:12.1.0-devel-ubuntu22.04@sha256:e3a8f7b933e77ecee74731198a2a5483e965b585cea2660675cf4bb152237e9b

4 DONE 0.0s

5 [ 4/10] RUN pip3 install "torch==2.1.1"

5 CACHED

6 [ 2/10] WORKDIR /workspace

6 CACHED

7 [ 3/10] RUN apt update && apt install -y python3-pip python3-packaging git ninja-build && pip3 install -U pip

7 CACHED

8 [ 5/10] RUN pip3 install "git+https://github.com/stanford-futuredata/megablocks.git"

8 CACHED

9 [internal] load build context

9 transferring context: 35B done

9 DONE 0.0s

10 [ 6/10] RUN pip3 install -v "git+https://github.com/vllm-project/vllm.git"

[....]

10 288.1 Compiling objects...

10 288.1 Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)

10 314.7 [1/10] /usr/local/cuda/bin/nvcc -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /tmp/pip-req-build-vx02uwf7/csrc/cuda_utils_kernels.cu -o /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/cuda_utils_kernels.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0

10 3687.0 [2/10] /usr/local/cuda/bin/nvcc -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /tmp/pip-req-build-vx02uwf7/csrc/activation_kernels.cu -o /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/activation_kernels.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0

10 3687.0 FAILED: /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/activation_kernels.o

10 3687.0 /usr/local/cuda/bin/nvcc -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /tmp/pip-req-build-vx02uwf7/csrc/activation_kernels.cu -o /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/activation_kernels.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0

10 3687.0 Killed

10 4196.7 [3/10] /usr/local/cuda/bin/nvcc -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /tmp/pip-req-build-vx02uwf7/csrc/pos_encoding_kernels.cu -o /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/pos_encoding_kernels.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0

10 4196.7 FAILED: /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/pos_encoding_kernels.o

10 4196.7 /usr/local/cuda/bin/nvcc -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /tmp/pip-req-build-vx02uwf7/csrc/pos_encoding_kernels.cu -o /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/pos_encoding_kernels.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0

10 4196.7 Killed

10 4520.8 [4/10] /usr/local/cuda/bin/nvcc -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /tmp/pip-req-build-vx02uwf7/csrc/cache_kernels.cu -o /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/cache_kernels.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0

10 4520.8 FAILED: /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/cache_kernels.o

10 4520.8 /usr/local/cuda/bin/nvcc -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /tmp/pip-req-build-vx02uwf7/csrc/cache_kernels.cu -o /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/cache_kernels.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0

10 4520.8 Killed

10 4604.5 [5/10] /usr/local/cuda/bin/nvcc -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /tmp/pip-req-build-vx02uwf7/csrc/layernorm_kernels.cu -o /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/layernorm_kernels.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0

10 4604.5 FAILED: /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/layernorm_kernels.o

10 4604.5 /usr/local/cuda/bin/nvcc -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /tmp/pip-req-build-vx02uwf7/csrc/layernorm_kernels.cu -o /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/layernorm_kernels.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0

10 4604.5 Killed

10 4773.3 [6/10] /usr/local/cuda/bin/nvcc -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /tmp/pip-req-build-vx02uwf7/csrc/quantization/awq/gemm_kernels.cu -o /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/quantization/awq/gemm_kernels.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0

10 4773.3 FAILED: /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/quantization/awq/gemm_kernels.o

10 4773.3 /usr/local/cuda/bin/nvcc -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-myvapo8z/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /tmp/pip-req-build-vx02uwf7/csrc/quantization/awq/gemm_kernels.cu -o /tmp/pip-req-build-vx02uwf7/build/temp.linux-x86_64-cpython-310/csrc/quantization/awq/gemm_kernels.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0

10 4773.3 Killed

ERROR: failed to receive status: rpc error: code = Unavailable desc = error reading from server: EOF

log.txt

ykacer commented 6 months ago

Hi @Good-Coffee, any idea on how to solve this issue? thanks

Good-Coffee commented 6 months ago

I didn't have a CUDA-capable graphics card

ykacer commented 6 months ago

clear, thanks.

ahmedalbahnasawi commented 5 months ago

replacing pip3 install -v "git+https://github.com/vllm-project/vllm.git with pip install vllm solve the issue.