search

NVIDIA / TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
https://nvidia.github.io/TensorRT-LLM
Apache License 2.0
8.57k stars 973 forks source link

Tensorrt_llm install issues #1129

Open KarihaZach opened 8 months ago

KarihaZach commented 8 months ago

System Info

When i install tensorrt_llm and type follow code : pip install tensorrt_llm --extra-index-url https://pypi.nvidia.com --extra-index-url https://download.pytorch.org/whl/cu121 it returns these error: Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com, https://download.pytorch.org/whl/cu121 Collecting tensorrt_llm Downloading tensorrt-llm-0.7.1.tar.gz (6.9 kB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [6 lines of output] Traceback (most recent call last): File "", line 2, in File "", line 34, in File "C:\Users\KarihaZach\AppData\Local\Temp\pip-install-hqtiy0_9\tensorrt-llm_f9311b1469704ce2bde863e0350fbfed\setup.py", line 90, in raise RuntimeError("Bad params") RuntimeError: Bad params [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details. The environment of my computer is win10,cudnn9.0,cuda12.2.2,python 3.10.11.and i was already installed Microsoft MPI(including two file),Every time when i run the wheel,the error always stuck in the "Bad params". Please help me to handle this,thanks a lot!

Who can help?

@juney-nvidia;@ncomly-nvidia;@kaiyux

wimueller commented 8 months ago

Are you sure you are using Python 3.10?

nignal commented 8 months ago

same problem.

Edit: changed python3.12 to 3.10 and working now.

pfldy2850 commented 8 months ago

same here.

I cannot install this package on ubuntu 22.04.

Shixiaowei02 commented 7 months ago

same here.

I cannot install this package on ubuntu 22.04.

Are you using Python 3.10?

stephanbertl commented 7 months ago

Same here, python 3.10.12 on Ubuntu 22.04.

Install of latest tensorrt_llm-0.9.0.dev2024031200-cp310-cp310-linux_x86_64.whl gives "bad params", previous version works fine.

aimestereo commented 2 months ago

for me it failes on 3.10.12 and works on 3.10.11 (Docker python:3.10.11-slim-buster)