some error happend when installing vllm

[finylink]

Nvidia jetson is aarch64 , in ubuntu20.04 server(cuda 12.2), when run "pip install vllm " , some error happened :

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [19 lines of output] /tmp/pip-build-env-jkd2f4g4/overlay/lib/python3.10/site-packages/torch/nn/modules/transformer.py:20: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:84.) device: torch.device = torch.device(torch._C._get_default_device()), # torch.device('cpu'), No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda' Traceback (most recent call last): File "/root/miniconda3/envs/python310/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in main() File "/root/miniconda3/envs/python310/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/root/miniconda3/envs/python310/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) File "/tmp/pip-build-env-jkd2f4g4/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel return self._get_build_requires(config_settings, requirements=['wheel']) File "/tmp/pip-build-env-jkd2f4g4/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires self.run_setup() File "/tmp/pip-build-env-jkd2f4g4/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 311, in run_setup exec(code, locals()) File "", line 442, in File "", line 404, in get_vllm_version NameError: name 'nvcc_cuda_version' is not defined. Did you mean: 'cuda_version'? [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[finylink]

some body help?

[thesues]

No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'

Did you install NVIDIA toolkit? because vllm requires torch == 2.1.2. so the nvidia toolkit should be at least 12.1 .

you can use

nvcc --verison

to check

[likenamehaojie]

i have the same problem in windows

[senruiYao]

I have the same problem, some body help?

[thesues]

I think you can check the env first:

python -c "import torch;from torch.utils.cpp_extension import CUDA_HOME; print(torch.version.cuda, CUDA_HOME)"

my output is

12.1 /usr/local/cuda-12.1

It means pytorch knows the runtime cuda version and the location of nvcc.

and then can you git clone this vllm repo? and manually install vllm and post the error log?

pip install -r requirements.txt
python setup install

BTW: vllm does not support ms windows by far.