search

NVIDIA / TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
https://nvidia.github.io/TensorRT-LLM
Apache License 2.0
7.7k stars 837 forks source link

error while installing pip3 install tensorrt_llm -U --extra-index-url https://pypi.nvidia.com #1198

Open safdar23 opened 5 months ago

safdar23 commented 5 months ago

System Info

tensorrt-llm installation fails on NVIDIA A100 with below error, i have Cuda 12.2

Collecting tensorrt_llm Using cached tensorrt-llm-0.8.0.tar.gz (6.9 kB) ERROR: Command errored out with exit status 1: command: /anaconda/envs/azureml_py38/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_mkl9ytt/tensorrt-llm/setup.py'"'"'; file='"'"'/tmp/pip-install-_mkl9ytt/tensorrt-llm/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-l8z5l156 cwd: /tmp/pip-install-_mkl9ytt/tensorrt-llm/ Complete output (5 lines): Traceback (most recent call last): File "", line 1, in File "/tmp/pip-install-_mkl9ytt/tensorrt-llm/setup.py", line 90, in raise RuntimeError("Bad params") RuntimeError: Bad params

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Who can help?

No response

Information

Tasks

Reproduction

run the command pip3 install tensorrt_llm -U --extra-index-url https://pypi.nvidia.com

Expected behavior

it should install tensorrt-llm

actual behavior

the above error is observed

additional notes

NA

byshiue commented 4 months ago

Could you try installing the requirements-dev.txt and requirements.txt first?

safdar23 commented 4 months ago

Thank you , i ran requirements-dev.txt as this calls requirements.txt and got the below error, please note as per my nvidia-smi output i updated to CUDA12.2. nvcc --version command also outputs the CUDA 12.2 version.

Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu121, https://pypi.nvidia.com/ Collecting accelerate==0.25.0 Downloading accelerate-0.25.0-py3-none-any.whl (265 kB) |████████████████████████████████| 265 kB 6.1 MB/s eta 0:00:01 Collecting build Downloading build-1.1.1-py3-none-any.whl (19 kB) Collecting colored Downloading colored-1.4.4.tar.gz (36 kB) Collecting cuda-python Downloading cuda_python-12.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (23.5 MB) |████████████████████████████████| 23.5 MB 100.1 MB/s eta 0:00:01 Collecting diffusers==0.15.0 Downloading diffusers-0.15.0-py3-none-any.whl (851 kB) |████████████████████████████████| 851 kB 93.2 MB/s eta 0:00:01 Collecting lark Downloading lark-1.1.9-py3-none-any.whl (111 kB) |████████████████████████████████| 111 kB 122.6 MB/s eta 0:00:01 Collecting mpi4py Downloading mpi4py-3.1.5.tar.gz (2.5 MB) |████████████████████████████████| 2.5 MB 116.8 MB/s eta 0:00:01 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done Requirement already satisfied: numpy in /anaconda/envs/azureml_py38/lib/python3.8/site-packages (from -r requirements.txt (line 10)) (1.21.6) Requirement already satisfied: onnx>=1.12.0 in /anaconda/envs/azureml_py38/lib/python3.8/site-packages (from -r requirements.txt (line 11)) (1.13.1) Collecting polygraphy Downloading polygraphy-0.49.0-py2.py3-none-any.whl (327 kB) |████████████████████████████████| 327 kB 100.5 MB/s eta 0:00:01 Requirement already satisfied: psutil in /anaconda/envs/azureml_py38/lib/python3.8/site-packages (from -r requirements.txt (line 13)) (5.9.5) Requirement already satisfied: pynvml>=11.5.0 in /anaconda/envs/azureml_py38/lib/python3.8/site-packages (from -r requirements.txt (line 14)) (11.5.0) Requirement already satisfied: sentencepiece>=0.1.99 in /anaconda/envs/azureml_py38/lib/python3.8/site-packages (from -r requirements.txt (line 15)) (0.1.99) Collecting tensorrt==9.2.0.post12.dev5 Downloading tensorrt-9.2.0.post12.dev5.tar.gz (6.8 kB) ERROR: Command errored out with exit status 1: command: /anaconda/envs/azureml_py38/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-6mt80x84/tensorrt/setup.py'"'"'; file='"'"'/tmp/pip-install-6mt80x84/tensorrt/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-rz5agf4t cwd: /tmp/pip-install-6mt80x84/tensorrt/ Complete output (15 lines): Traceback (most recent call last): File "", line 1, in File "/tmp/pip-install-6mt80x84/tensorrt/setup.py", line 137, in raise RuntimeError(open("ERROR.txt", "r").read()) RuntimeError: ########################################################################################### The package you are trying to install is only a placeholder project on PyPI.org repository. This package is hosted on NVIDIA Python Package Index.

This package can be installed as:
```
$ pip install --no-cache-dir --extra-index-url https://pypi.nvidia.com/ tensorrt
```
###########################################################################################

----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Shixiaowei02 commented 4 months ago

Please install Python 3.10 first, thanks!