search

NVIDIA / TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
https://nvidia.github.io/TensorRT-LLM
Apache License 2.0
7.34k stars 794 forks source link

ModuleNotFoundError: No module named 'tensorrt_llm.bindings.BuildInfo' #1831

Closed PoTaTo-Mika closed 2 days ago

PoTaTo-Mika commented 4 days ago

When building conda env, I met with this problem. I installed TensorRT by using source code from NVIDIA official Website,and used pip install -r requirement.txt in main and /examples/qwen, when it all finished, I found it unable to import tensorrt_llm.

Details

Traceback (most recent call last): File "", line 1, in File "/home/hjzd/miniconda3/envs/TRT/lib/python3.11/site-packages/tensorrt_llm/init.py", line 32, in import tensorrt_llm.functional as functional File "/home/hjzd/miniconda3/envs/TRT/lib/python3.11/site-packages/tensorrt_llm/functional.py", line 28, in from . import graph_rewriting as gw File "/home/hjzd/miniconda3/envs/TRT/lib/python3.11/site-packages/tensorrt_llm/graph_rewriting.py", line 12, in from .network import Network File "/home/hjzd/miniconda3/envs/TRT/lib/python3.11/site-packages/tensorrt_llm/network.py", line 27, in from tensorrt_llm.module import Module File "/home/hjzd/miniconda3/envs/TRT/lib/python3.11/site-packages/tensorrt_llm/module.py", line 17, in from ._common import default_net File "/home/hjzd/miniconda3/envs/TRT/lib/python3.11/site-packages/tensorrt_llm/_common.py", line 31, in from ._utils import str_dtype_to_trt File "/home/hjzd/miniconda3/envs/TRT/lib/python3.11/site-packages/tensorrt_llm/_utils.py", line 30, in from tensorrt_llm.bindings.BuildInfo import ENABLE_MULTI_DEVICE ModuleNotFoundError: No module named 'tensorrt_llm.bindings.BuildInfo'

hijkzzz commented 4 days ago

Please use the docker container See the tutorial: https://nvidia.github.io/TensorRT-LLM/installation/linux.html