Open Hukongtao opened 1 week ago
Can you provide the content of the handler_trt.py
? Please give a simple code sample related to TensorRT-LLM for reproduction.
Can you provide the content of the
handler_trt.py
? Please give a simple code sample related to TensorRT-LLM for reproduction.
I found that this error has nothing to do with handler_trt.py. I installed trt-llm in the following way and got the error.
pip3 install --no-cache-dir tensorrt_llm -U --pre --extra-index-url https://pypi.nvidia.com
But when I installed trt-llm from source, the error disappeared
git clone https://github.com/NVIDIA/TensorRT-LLM TensorRT-LLM-py39
git checkout 75057cd036af25e288c004d8ac9e52fd2d6224aa
git submodule update --init --recursive
git lfs install --force
git lfs pull
python3 ./scripts/build_wheel.py --trt_root /usr/local/tensorrt/ -c
I queried the information for the commit 75057cd036af25e288c004d8ac9e52fd2d6224aa
, it has the following information:
commit 75057cd036af25e288c004d8ac9e52fd2d6224aa (HEAD -> main, origin/main, origin/HEAD)
Author: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
Date: Tue Oct 15 15:28:40 2024 +0800
So it is the latest bi-weekly release.
Can you share the version of the tensorrt_llm pip installed? Just get the input of import tensorrt_llm; print(tensorrt_llm.__version__)
.
cc @kaiyux for viz.
System Info
Who can help?
No response
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
The problem occurs in this sentence: MpiComm.local_init()
Expected behavior
run successfully
actual behavior
hang up
additional notes
xxx