error when build TensorRT_LLM engine

NVIDIA / TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

https://nvidia.github.io/TensorRT-LLM

Apache License 2.0

8.81k stars 1.01k forks source link

error when build TensorRT_LLM engine #2124

Open zxy1119 opened 3 months ago

zxy1119 commented 3 months ago

System Info

CUDA 11.8 Please tell me what causes this error.

Who can help?

No response

Information

[ ] The official example scripts
[ ] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction

python scripts/build_wheel.py --clean --trt_root /data0/baolili/TensorRT-10.1.0.27 --python_bindings --benchmarks

Expected behavior

Successfully built tensor_llm

actual behavior

Failed to build

additional notes

Failed to build

Kefeng-Duan commented 3 months ago

Hi, @zxy1119 Which GPU are you using?

zxy1119 commented 3 months ago

@Kefeng-Duan nvidia A800 i upgraded CUDA version 11.8 -> 12.2 but still build tensorrt_llm error

Kefeng-Duan commented 3 months ago

@zxy1119 have you clean the workspace?

zxy1119 commented 3 months ago

i use python3 scripts/build_wheel.py --clean --trt_root /usr/local/TensorRT-10.1.0.27,but and it is no *.whl file generated in the directory cpp/build. how to clean the workspace?

Kefeng-Duan commented 3 months ago

@zxy1119 could you update to the latest version, delete the build folder and rerun the build wheel command?

zxy1119 commented 3 months ago

it is the latest version , and i think the problem is cutlass_kernels

this is my version 93546f25e777541f6b2d571dd1a9a6d9906094d2 082e3c8ef4a1ea6d619ef2e373a8b6308f36b12f b203a707c6c7477c21bb9e77eeb4f2cdadd18a60

Kefeng-Duan commented 3 months ago

@zxy1119 could you show more screenshot about the cutlass error?

zxy1119 commented 3 months ago

sure, this is my build log cmake_log.log

zxy1119 commented 3 months ago

This part of the code seems to be the issue.

yuhengxnv commented 3 months ago

@zxy1119 GCC 8 seems too old, please try with a newer GCC. I tested and failed with 8.3, succeeded with 12.2, FYI.

github-actions[bot] commented 1 month ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 15 days."