python setup.py develop did not run successfully

[heraistudios]

I ran the following command:

docker build -t <<IMAGE_NAME>> --build-arg MODEL_NAME="TheBloke_vicuna-7B-1.1-GPTQ" --build-arg MODEL_BASE_PATH="/models" .

On my windows system using WSL. My computer has an NVIDIA GPU, however when I try to create a docker image using the command above, I get the following error:

207.2   Running setup.py develop for vllm
466.3     error: subprocess-exited-with-error
466.3
466.3     × python setup.py develop did not run successfully.
466.3     │ exit code: 1
466.3     ╰─> [180 lines of output]
466.3         No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
466.3         running develop
466.3         /usr/local/lib/python3.11/dist-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.
466.3         !!
466.3
466.3                 ********************************************************************************
466.3                 Please avoid running ``setup.py`` and ``easy_install``.
466.3                 Instead, use pypa/build, pypa/installer or other
466.3                 standards-based tools.
466.3
466.3                 See https://github.com/pypa/setuptools/issues/917 for details.
466.3                 ********************************************************************************
466.3
466.3         !!
466.3           easy_install.initialize_options(self)
466.3         /usr/local/lib/python3.11/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
466.3         !!
466.3
466.3                 ********************************************************************************
466.3                 Please avoid running ``setup.py`` directly.
466.3                 Instead, use pypa/build, pypa/installer or other
466.3                 standards-based tools.
466.3
466.3                 See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
466.3                 ********************************************************************************
466.3
466.3         !!
466.3           self.initialize_options()
466.3         running egg_info
466.3         creating vllm.egg-info
466.3         writing vllm.egg-info/PKG-INFO
466.3         writing dependency_links to vllm.egg-info/dependency_links.txt
466.3         writing requirements to vllm.egg-info/requires.txt
466.3         writing top-level names to vllm.egg-info/top_level.txt
466.3         writing manifest file 'vllm.egg-info/SOURCES.txt'
466.3         reading manifest file 'vllm.egg-info/SOURCES.txt'
466.3         reading manifest template 'MANIFEST.in'
466.3         adding license file 'LICENSE'
466.3         writing manifest file 'vllm.egg-info/SOURCES.txt'
466.3         running build_ext
466.3         /usr/local/lib/python3.11/dist-packages/torch/utils/cpp_extension.py:424: UserWarning: There are no x86_64-linux-gnu-g++ version bounds defined for CUDA version 11.8
466.3           warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
466.3         building 'vllm._C' extension
466.3         creating /src/vllm/build
466.3         creating /src/vllm/build/temp.linux-x86_64-cpython-311
466.3         creating /src/vllm/build/temp.linux-x86_64-cpython-311/csrc
466.3         creating /src/vllm/build/temp.linux-x86_64-cpython-311/csrc/attention
466.3         creating /src/vllm/build/temp.linux-x86_64-cpython-311/csrc/quantization
466.3         creating /src/vllm/build/temp.linux-x86_64-cpython-311/csrc/quantization/awq
466.3         creating /src/vllm/build/temp.linux-x86_64-cpython-311/csrc/quantization/gptq
466.3         creating /src/vllm/build/temp.linux-x86_64-cpython-311/csrc/quantization/squeezellm
466.3         Emitting ninja build file /src/vllm/build/temp.linux-x86_64-cpython-311/build.ninja...
466.3         Compiling objects...

This if the full log output: logs.txt

Any help would be greatly appreciated!!!

[vividias]

I'm having the exactly same error :/ also using wsl in windows with GPU

[bartlettD]

Try adding MAX_JOBS=1 to your docker environment. I saw this when trying to compile on a runner and it looks like an Out of Memory kill because I can't see any real errors in the log.

[alpayariyak]

In the latest version, we have changed the base image to one that already has vLLM compiled, which should solve this problem.