[Installation]: vllm install error in jetson agx orin

[FanZhang91]

Your current environment

OS: Ubuntu 22.04 jammy Kernel: aarch64 Linux 5.15.136-tegra Uptime: 1d 23h 22m Shell: bash 5.1.16 Disk: 137G / 1.8T (8%) CPU: ARM Cortex-A78AE @ 12x 2.2016GHz GPU: Orin (nvgpu) RAM: 3162MiB / 62841MiB Nova Config: nova-devkit Nova Version: 1.3.0 Jetpack: 6.0+b106 torch: 2.3.0(use pytorch aarch64 wheels)

How you are installing vllm

I want to install vllm in Jetson AGX Orin. I use the script "pip install vllm" and try to install from source "pip install -e .", but both these method return failture.

############################# error info ################################ Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple/ Collecting vllm Using cached https://pypi.tuna.tsinghua.edu.cn/packages/1c/e2/b437c1d5a39814acfc706e714eceee7fe80455a9666a13b7582205b30345/vllm-0.5.4.tar.gz (958 kB) Installing build dependencies ... done Getting requirements to build wheel ... error error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [21 lines of output] /tmp/pip-build-env-94kz6585/overlay/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py:258: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:84.) cpu = _conversion_method_template(device=torch.device("cpu")) fatal: not a git repository (or any of the parent directories): .git

:56: RuntimeWarning: Failed to get commit hash: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128. Traceback (most recent call last): File "/home/xxx/miniconda3/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in main() File "/home/xxx/miniconda3/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/home/xxx/miniconda3/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) File "/tmp/pip-build-env-94kz6585/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel return self._get_build_requires(config_settings, requirements=[]) File "/tmp/pip-build-env-94kz6585/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires self.run_setup() File "/tmp/pip-build-env-94kz6585/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 313, in run_setup exec(code, locals()) File "", line 458, in File "", line 379, in get_vllm_version RuntimeError: Unknown runtime environment [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. [notice] A new release of pip is available: 24.1.2 -> 24.2 [notice] To update, run: pip install --upgrade pip error: subprocess-exited-with-error × Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip.

[youkaichao]

might because you are using arm64 platform, which is not supported directly yet.

see also https://github.com/vllm-project/vllm/issues/2021

[FanZhang91]

might because you are using arm64 platform, which is not supported directly yet.

see also #2021

Is there a plan to support VLLM under ARM architecture, or is there any roundabout solution?

[shahizat]

hi, also interested to run vLLM on the Nvidia AGX orin dev kit.

[conroy-cheers]

Looking to add/restore Jetson support in #9735