search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
29.55k stars 4.44k forks source link

[Bug]: On a machine with an A100 GPU, when running the Dockerfile of version 0.5.5, an error occurs. #7914

Closed zjjznw123 closed 2 months ago

zjjznw123 commented 2 months ago

Your current environment

On a machine with an A100 GPU Dockerfile 0.5.5

🐛 Describe the bug

64 running build_ext

34 3.401 Using MAX_JOBS=8 as the number of jobs.

34 3.405 Using NVCC_THREADS=8 as the number of nvcc threads.

34 3.660 -- The CXX compiler identification is GNU 11.4.0

34 3.730 -- Detecting CXX compiler ABI info

34 4.065 -- Detecting CXX compiler ABI info - done

34 4.090 -- Check for working CXX compiler: /usr/bin/c++ - skipped

34 4.090 -- Detecting CXX compile features

34 4.091 -- Detecting CXX compile features - done

34 4.092 -- Build type: Release

34 4.092 -- Target device: cuda

34 4.280 -- Found Python: /usr/bin/python3 (found version "3.10.12") found components: Interpreter Development.Module Development.SABIModule

34 4.280 -- Found python matching: /usr/bin/python3.

34 6.273 -- Found CUDA: /usr/local/cuda (found version "12.4")

34 7.528 -- The CUDA compiler identification is NVIDIA 12.4.131

34 7.540 -- Detecting CUDA compiler ABI info

34 8.908 -- Detecting CUDA compiler ABI info - done

34 8.998 -- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped

34 9.027 -- Detecting CUDA compile features

34 9.028 -- Detecting CUDA compile features - done

34 9.036 -- Found CUDAToolkit: /usr/local/cuda/include (found version "12.4.131")

34 9.043 -- Performing Test CMAKE_HAVE_LIBC_PTHREAD

34 9.347 -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success

34 9.351 -- Found Threads: TRUE

34 9.383 -- Caffe2: CUDA detected: 12.4

34 9.383 -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc

34 9.383 -- Caffe2: CUDA toolkit directory: /usr/local/cuda

34 9.731 -- Caffe2: Header version is: 12.4

34 9.971 -- /usr/local/cuda/lib64/libnvrtc.so shorthash is 6d168ef8

34 9.972 -- USE_CUDNN is set to 0. Compiling without cuDNN support

34 9.972 -- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support

34 9.972 CMake Warning at /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/public/utils.cmake:382 (message):

34 9.972 In the future we will require one to explicitly pass TORCH_CUDA_ARCH_LIST

34 9.972 to cmake instead of implicitly setting it as an env variable. This will

34 9.972 become a FATAL_ERROR in future version of pytorch.

34 9.972 Call Stack (most recent call first):

34 9.972 /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/public/cuda.cmake:300 (torch_cuda_get_nvcc_gencode_flag)

34 9.972 /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)

34 9.972 /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)

34 9.972 CMakeLists.txt:70 (find_package)

34 9.972

34 9.972

34 9.973 -- Added CUDA NVCC flags for: -gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_89,code=sm_89;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_90,code=compute_90

34 9.993 CMake Warning at /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):

34 9.993 static library kineto_LIBRARY-NOTFOUND not found.

34 9.993 Call Stack (most recent call first):

34 9.993 /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:120 (append_torchlib_if_found)

34 9.993 CMakeLists.txt:70 (find_package)

34 9.993

34 9.993

34 9.994 -- Found Torch: /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch.so

34 9.995 -- Enabling core extension.

34 9.995 -- CUDA supported arches: 7.0;7.5;8.0;8.6;8.9;9.0

34 9.996 -- CUDA target arches: 70-real;75-real;80-real;86-real;89-real;90-real;90-virtual

34 145.6 -- CMake Version: 3.30.2

34 145.6 -- CUTLASS 3.5.1

34 145.6 -- CUDART: /usr/local/cuda/lib64/libcudart.so

34 145.6 -- CUDA Driver: /usr/local/cuda/lib64/stubs/libcuda.so

34 145.6 -- NVRTC: /usr/local/cuda/lib64/libnvrtc.so

34 145.6 -- Default Install Location: install

34 145.8 -- Found Python3: /usr/bin/python3.10 (found suitable version "3.10.12", minimum required is "3.5") found components: Interpreter

34 145.8 -- Make cute::tuple be the new standard-layout tuple type

34 145.8 -- CUDA Compilation Architectures: 70;72;75;80;86;87;89;90;90a

34 145.8 -- Enable caching of reference results in conv unit tests

34 145.8 -- Enable rigorous conv problem sizes in conv unit tests

34 145.8 -- Using NVCC flags: --expt-relaxed-constexpr;-DCUTE_USE_PACKED_TUPLE=1;-DCUTLASS_TEST_LEVEL=0;-DCUTLASS_TEST_ENABLE_CACHED_RESULTS=1;-DCUTLASS_CONV_UNIT_TEST_RIGOROUS_SIZE_ENABLED=1;-DCUTLASS_DEBUG_TRACE_LEVEL=0;-Xcompiler=-Wconversion;-Xcompiler=-fno-strict-aliasing

34 145.9 fatal: not a git repository (or any of the parent directories): .git

34 145.9 -- CUTLASS Revision: Unable to detect, Git returned code 128.

34 145.9 -- Configuring cublas ...

34 145.9 -- cuBLAS Disabled.

34 145.9 -- Configuring cuBLAS ... done.

34 146.2 -- Machete generation completed successfully.

34 146.2 -- Machete generated sources: /workspace/csrc/quantization/machete/generated/machete_mm_bf16u4.cu;/workspace/csrc/quantization/machete/generated/machete_mm_bf16u4_impl_part0.cu;/workspace/csrc/quantization/machete/generated/machete_mm_bf16u4_impl_part1.cu;/workspace/csrc/quantization/machete/generated/machete_mm_bf16u4b8.cu;/workspace/csrc/quantization/machete/generated/machete_mm_bf16u4b8_impl_part0.cu;/workspace/csrc/quantization/machete/generated/machete_mm_bf16u4b8_impl_part1.cu;/workspace/csrc/quantization/machete/generated/machete_mm_bf16u8.cu;/workspace/csrc/quantization/machete/generated/machete_mm_bf16u8_impl_part0.cu;/workspace/csrc/quantization/machete/generated/machete_mm_bf16u8_impl_part1.cu;/workspace/csrc/quantization/machete/generated/machete_mm_bf16u8b128.cu;/workspace/csrc/quantization/machete/generated/machete_mm_bf16u8b128_impl_part0.cu;/workspace/csrc/quantization/machete/generated/machete_mm_bf16u8b128_impl_part1.cu;/workspace/csrc/quantization/machete/generated/machete_mm_f16u4.cu;/workspace/csrc/quantization/machete/generated/machete_mm_f16u4_impl_part0.cu;/workspace/csrc/quantization/machete/generated/machete_mm_f16u4_impl_part1.cu;/workspace/csrc/quantization/machete/generated/machete_mm_f16u4b8.cu;/workspace/csrc/quantization/machete/generated/machete_mm_f16u4b8_impl_part0.cu;/workspace/csrc/quantization/machete/generated/machete_mm_f16u4b8_impl_part1.cu;/workspace/csrc/quantization/machete/generated/machete_mm_f16u8.cu;/workspace/csrc/quantization/machete/generated/machete_mm_f16u8_impl_part0.cu;/workspace/csrc/quantization/machete/generated/machete_mm_f16u8_impl_part1.cu;/workspace/csrc/quantization/machete/generated/machete_mm_f16u8b128.cu;/workspace/csrc/quantization/machete/generated/machete_mm_f16u8b128_impl_part0.cu;/workspace/csrc/quantization/machete/generated/machete_mm_f16u8b128_impl_part1.cu;/workspace/csrc/quantization/machete/generated/machete_prepack_bf16u4.cu;/workspace/csrc/quantization/machete/generated/machete_prepack_bf16u4b8.cu;/workspace/csrc/quantization/machete/generated/machete_prepack_bf16u8.cu;/workspace/csrc/quantization/machete/generated/machete_prepack_bf16u8b128.cu;/workspace/csrc/quantization/machete/generated/machete_prepack_f16u4.cu;/workspace/csrc/quantization/machete/generated/machete_prepack_f16u4b8.cu;/workspace/csrc/quantization/machete/generated/machete_prepack_f16u8.cu;/workspace/csrc/quantization/machete/generated/machete_prepack_f16u8b128.cu

34 146.2 -- Enabling C extension.

34 146.2 -- Enabling moe extension.

34 146.2 -- Configuring done (142.7s)

34 146.3 -- Generating done (0.1s)

34 146.3 -- Build files have been written to: /workspace/build/temp.linux-x86_64-cpython-310

34 146.4 Using MAX_JOBS=8 as the number of jobs.

34 146.4 Using NVCC_THREADS=8 as the number of nvcc threads.

34 147.0 [1/66] Building CXX object CMakeFiles/_core_C.dir/csrc/core/torch_bindings.cpp.o

34 147.0 FAILED: CMakeFiles/_core_C.dir/csrc/core/torch_bindings.cpp.o

34 147.0 sccache /usr/bin/c++ -DPy_LIMITED_API=3 -DTORCH_EXTENSION_NAME=_core_C -DUSE_C10D_GLOO -DUSE_C10D_NCCL -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_core_C_EXPORTS -I/workspace/csrc -isystem /usr/include/python3.10 -isystem /usr/local/lib/python3.10/dist-packages/torch/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -isystem /usr/local/cuda/include -O3 -DNDEBUG -fPIC -D_GLIBCXX_USE_CXX11_ABI=0 -MD -MT CMakeFiles/_core_C.dir/csrc/core/torch_bindings.cpp.o -MF CMakeFiles/_core_C.dir/csrc/core/torch_bindings.cpp.o.d -o CMakeFiles/_core_C.dir/csrc/core/torch_bindings.cpp.o -c /workspace/csrc/core/torch_bindings.cpp

34 147.0 [2024-08-27T13:58:57Z DEBUG sccache::config] Attempting to read config file at "/root/.config/sccache/config"

34 147.0 [2024-08-27T13:58:57Z DEBUG sccache::config] Couldn't open config file: failed to open file /root/.config/sccache/config

34 147.0 [2024-08-27T13:58:57Z DEBUG sccache::config] Attempting to read config file at "/root/.config/sccache/config"

34 147.0 [2024-08-27T13:58:57Z DEBUG sccache::config] Couldn't open config file: failed to open file /root/.config/sccache/config

34 147.0 [2024-08-27T13:58:57Z INFO sccache::server] start_server: port: 4226

34 147.0 [2024-08-27T13:58:57Z INFO sccache::server] No scheduler address configured, disabling distributed sccache

34 147.0 [2024-08-27T13:58:57Z DEBUG sccache::cache::cache] Init s3 cache with bucket vllm-build-sccache, endpoint None

34 147.0 [2024-08-27T13:58:57Z DEBUG opendal::services::s3::backend] backend build started: S3Builder { config: S3Config { root: None, bucket: "vllm-build-sccache", endpoint: None, region: Some("us-west-2"), .. }, .. }

34 147.0 [2024-08-27T13:58:57Z DEBUG opendal::services::s3::backend] backend use root /

34 147.0 [2024-08-27T13:58:57Z DEBUG opendal::services::s3::backend] backend use bucket vllm-build-sccache

34 147.0 [2024-08-27T13:58:57Z DEBUG reqsign::aws::config] load_via_profile_config_file failed: No such file or directory (os error 2)

34 147.0

34 147.0 Stack backtrace:

34 147.0 0:

34 147.0 1:

34 147.0 2:

34 147.0 3:

34 147.0 4:

34 147.0 5:

34 147.0 6:

34 147.0 7:

34 147.0 8:

34 147.0 [2024-08-27T13:58:57Z DEBUG reqsign::aws::config] load_via_profile_shared_credentials_file failed: No such file or directory (os error 2)

34 147.0

34 147.0 Stack backtrace:

34 147.0 0:

34 147.0 1:

34 147.0 2:

34 147.0 3:

34 147.0 4:

34 147.0 5:

34 147.0 6:

34 147.0 7:

34 147.0 8:

34 147.0 [2024-08-27T13:58:57Z DEBUG opendal::services::s3::backend] backend use region: us-west-2

34 147.0 [2024-08-27T13:58:57Z DEBUG opendal::services::s3::backend] backend use endpoint: https://s3.us-west-2.amazonaws.com/vllm-build-sccache

34 147.0 [2024-08-27T13:58:57Z DEBUG opendal::services::s3::backend] backend build finished

34 147.0 [2024-08-27T13:58:57Z DEBUG opendal::services] service=s3 operation=metadata -> started

34 147.0 [2024-08-27T13:58:57Z DEBUG opendal::services] service=s3 operation=metadata -> finished: AccessorInfo { scheme: S3, root: "/", name: "vllm-build-sccache", native_capability: { Stat | Read | Write | Delete | Copy | List | Presign | Batch }, full_capability: { Stat | Read | Write | CreateDir | Delete | Copy | List | Presign | Batch } }

34 147.0 [2024-08-27T13:58:57Z DEBUG opendal::services] service=s3 operation=stat path=.sccache_check -> started

34 147.0 [2024-08-27T13:58:57Z DEBUG hyper::proto::h1::io] flushed 183 bytes

34 147.0 [2024-08-27T13:58:57Z DEBUG hyper::proto::h1::io] parsed 5 headers

34 147.0 [2024-08-27T13:58:57Z DEBUG hyper::proto::h1::conn] incoming body is content-length (154 bytes)

34 147.0 [2024-08-27T13:58:57Z DEBUG hyper::proto::h1::conn] incoming body completed

34 147.0 [2024-08-27T13:58:57Z DEBUG reqsign::aws::credential] load credential via imds_v2 failed: request to AWS EC2 Metadata Services failed:

34 147.0

34 147.0 404 Not Found

34 147.0

34 147.0

34 147.0

404 Not Found

34 147.0 The resource could not be found.

34 147.0

34 147.0

34 147.0

34 147.0

34 147.0

34 147.0

34 147.0 Stack backtrace:

34 147.0 0:

34 147.0 1:

34 147.0 2:

34 147.0 3:

34 147.0 4:

34 147.0 5:

34 147.0 6:

34 147.0 7:

34 147.0 8:

34 147.0 9:

34 147.0 10:

34 147.0 11:

34 147.0 12:

34 147.0 13:

34 147.0 14:

34 147.0 15:

34 147.0 16:

34 147.0 17:

34 147.0 18:

34 147.0 19:

34 147.0 20:

34 147.0 21:

34 147.0 22:

34 147.0 23:

34 147.0 24:

34 147.0 25:

34 147.0 [2024-08-27T13:58:57Z WARN opendal::services] service=s3 operation=stat path=.sccache_check -> PermissionDenied (permanent) at stat, context: { service: s3, path: .sccache_check } => no valid credential found and anonymous access is not allowed

34 147.0 [2024-08-27T13:58:57Z ERROR sccache::server] storage check failed for: cache storage failed to read: PermissionDenied (permanent) at stat => no valid credential found and anonymous access is not allowed

34 147.0

34 147.0 Context:

34 147.0 service: s3

34 147.0 path: .sccache_check

34 147.0

34 147.0 Backtrace:

34 147.0 0:

34 147.0 1:

34 147.0 2:

34 147.0 3:

34 147.0 4:

34 147.0 5:

34 147.0 6:

34 147.0 7:

34 147.0 8:

34 147.0 9:

34 147.0 10:

34 147.0 11:

34 147.0 12:

34 147.0 13:

34 147.0 14:

34 147.0 15:

34 147.0 16:

34 147.0 17:

34 147.0 18:

34 147.0 19:

34 147.0 20:

34 147.0 21:

34 147.0

34 147.0

34 147.0

34 147.0 Stack backtrace:

34 147.0 0:

34 147.0 1:

34 147.0 2:

34 147.0 3:

34 147.0 4:

34 147.0 5:

34 147.0 6:

34 147.0 7:

34 147.0 8:

34 147.0 [2024-08-27T13:58:57Z DEBUG sccache::server] notify_server_startup(Err { reason: "cache storage failed to read: PermissionDenied (permanent) at stat => no valid credential found and anonymous access is not allowed\n\nContext:\n service: s3\n path: .sccache_check\n\nBacktrace:\n 0: \n 1: \n 2: \n 3: \n 4: \n 5: \n 6: \n 7: \n 8: \n 9: \n 10: \n 11: \n 12: \n 13: \n 14: \n 15: \n 16: \n 17: \n 18: \n 19: \n 20: \n 21: \n\n" })

34 147.0 sccache: error: Server startup failed: cache storage failed to read: PermissionDenied (permanent) at stat => no valid credential found and anonymous access is not allowed

34 147.0

34 147.0 Context:

34 147.0 service: s3

34 147.0 path: .sccache_check

34 147.0

34 147.0 Backtrace:

34 147.0 0:

34 147.0 1:

34 147.0 2:

34 147.0 3:

34 147.0 4:

34 147.0 5:

34 147.0 6:

34 147.0 7:

34 147.0 8:

34 147.0 9:

34 147.0 10:

34 147.0 11:

34 147.0 12:

34 147.0 13:

34 147.0 14:

34 147.0 15:

34 147.0 16:

34 147.0 17:

34 147.0 18:

34 147.0 19:

34 147.0 20:

34 147.0 21:

34 147.0

34 147.0

34 147.0 Run with SCCACHE_LOG=debug SCCACHE_NO_DAEMON=1 to get more information

34 147.0 sccache: error: cache storage failed to read: PermissionDenied (permanent) at stat => no valid credential found and anonymous access is not allowed

34 147.0

34 147.0 Context:

34 147.0 service: s3

34 147.0 path: .sccache_check

34 147.0

34 147.0 Backtrace:

34 147.0 0:

34 147.0 1:

34 147.0 2:

34 147.0 3:

34 147.0 4:

34 147.0 5:

34 147.0 6:

34 147.0 7:

34 147.0 8:

34 147.0 9:

34 147.0 10:

34 147.0 11:

34 147.0 12:

34 147.0 13:

34 147.0 14:

34 147.0 15:

34 147.0 16:

34 147.0 17:

34 147.0 18:

34 147.0 19:

34 147.0 20:

34 147.0 21:

34 147.0

34 147.0

34 147.0 ninja: build stopped: subcommand failed.

34 147.0 Traceback (most recent call last):

34 147.0 File "/workspace/setup.py", line 474, in

34 147.0 setup(

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/init.py", line 111, in setup

34 147.0 return distutils.core.setup(**attrs)

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 184, in setup

34 147.0 return run_commands(dist)

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 200, in run_commands

34 147.0 dist.run_commands()

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 964, in run_commands

34 147.0 self.run_command(cmd)

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 948, in run_command

34 147.0 super().run_command(command)

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 983, in run_command

34 147.0 cmd_obj.run()

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/command/bdist_wheel.py", line 384, in run

34 147.0 self.run_command("build")

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py", line 316, in run_command

34 147.0 self.distribution.run_command(command)

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 948, in run_command

34 147.0 super().run_command(command)

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 983, in run_command

34 147.0 cmd_obj.run()

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build.py", line 135, in run

34 147.0 self.run_command(cmd_name)

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py", line 316, in run_command

34 147.0 self.distribution.run_command(command)

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 948, in run_command

34 147.0 super().run_command(command)

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 983, in run_command

34 147.0 cmd_obj.run()

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/command/build_ext.py", line 96, in run

34 147.0 _build_ext.run(self)

34 147.0 File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 359, in run

34 147.0 self.build_extensions()

34 147.0 File "/workspace/setup.py", line 238, in build_extensions

34 147.0 subprocess.check_call(["cmake", *build_args], cwd=self.build_temp)

34 147.0 File "/usr/lib/python3.10/subprocess.py", line 369, in check_call

34 147.0 raise CalledProcessError(retcode, cmd)

34 147.0 subprocess.CalledProcessError: Command '['cmake', '--build', '.', '-j=1', '--target=_core_C', '--target=_moe_C', '--target=_C']' returned non-zero exit status 1.

executor failed running [/bin/sh -c if [ "$USE_SCCACHE" = "1" ]; then echo "Installing sccache..." && tar -xzf sccache.tar.gz && sudo mv sccache-v0.8.1-x86_64-unknown-linux-musl/sccache /usr/bin/sccache && rm -rf sccache.tar.gz sccache-v0.8.1-x86_64-unknown-linux-musl && export SCCACHE_BUCKET=${SCCACHE_BUCKET_NAME} && export SCCACHE_REGION=${SCCACHE_REGION_NAME} && export SCCACHE_IDLE_TIMEOUT=0 && export CMAKE_BUILD_TYPE=Release && export SCCACHE_LOG=debug && export SCCACHE_NO_DAEMON=1 && sccache --show-stats && python3 setup.py bdist_wheel --dist-dir=dist --py-limited-api=cp38 && sccache --show-stats; fi]: exit code: 1

Executing the command：DOCKER_BUILDKIT=1 docker build -t vllm0.5.5 -f Dockerfile0.5.5 . --build-arg USE_SCCACHE=1 --build-arg max_jobs=8

Before submitting a new issue...

youkaichao commented 2 months ago

https://github.com/vllm-project/vllm/pull/34 147.0 sccache: error: Server startup failed: cache storage failed to read: PermissionDenied (permanent) at stat => no valid credential found and anonymous access is not allowed https://github.com/vllm-project/vllm/pull/34 147.0 sccache: error: cache storage failed to read: PermissionDenied (permanent) at stat => no valid credential found and anonymous access is not allowed

cc @simon-mo @khluu does this mean our dockerfile needs to be built with credential?

simon-mo commented 2 months ago

USE_SCCACHE=1 should be turned off when not in CI system

zjjznw123 commented 2 months ago

USE_SCCACHE=1 should be turned off when not in CI system

What is a CI system? How can I modify it? Thank you. cc @simon-mo

simon-mo commented 2 months ago 
- DOCKER_BUILDKIT=1 docker build -t vllm0.5.5 -f Dockerfile0.5.5 . --build-arg USE_SCCACHE=1 --build-arg max_jobs=8
+ DOCKER_BUILDKIT=1 docker build -t vllm0.5.5 -f Dockerfile0.5.5 . --build-arg max_jobs=8
khluu commented 2 months ago

If you'd like to reuse our sccache bucket for faster build, you can also export SCCACHE_S3_NO_CREDENTIALS=0 (add it right after export SCCACHE_BUCKET=... in the Dockerbuild)