Seg fault after loaded models in official example

[LeatherDeerAU]

System Info

arch - x86-64 gpu - rtx3070 docker image nvcr.io/nvidia/tritonserver:24.01-trtllm-python-py3 tensorRT-LLM-backend tag - 0.7.2 tensorRT-LLM tag - 0.7.1 (80bc07510ac4ddf13c0d76ad295cdb2b75614618)

Who can help?

@juney-nvidia

Information

• [X] The official example scripts
• [ ] My own modified scripts

Tasks

• [X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

1. use nvcr.io/nvidia/tritonserver:24.01-trtllm-python-py3 image
2. clone tensorRT-LLM backend 0.7.2 tag
3. build tensorRT*.whl in container from tensorRT-LLM submodule in backend repo
4. try launch models from official example

Expected behavior

models uploaded successfully

actual behavior

sig_fault_logs.txt

additional notes

Which backend tag should I use for triton-container version 24-01?

related topics (?): https://github.com/triton-inference-server/tensorrtllm_backend/issues/273 https://github.com/NVIDIA/TensorRT-LLM/issues/782 https://github.com/triton-inference-server/tensorrtllm_backend/issues/88

[LeatherDeerAU]

Example models config.pbtxt files: postproccesing.txt preprocessing.txt tensorrt-llm.txt

[byshiue]

Could you share all scripts you use?