ailed to read text proto from tensorrtllm_backend/triton_model_repo/tensorrt_llm/config.pbtxt

System Info

[libprotobuf ERROR /tmp/tritonbuild/tritonserver/build/_deps/repo-third-party-build/grpc-repo/src/grpc/third_party/protobuf/src/google/protobuf/text_format.cc:335] Error parsing text-format inference.ModelConfig: 29:17: Expected integer, got: $ E0617 11:16:30.424716 31125 model_repository_manager.cc:1335] Poll failed for model directory 'tensorrt_llm': failed to read text proto from tensorrtllm_backend/triton_model_repo/tensorrt_llm/config.pbtxt

Who can help?

No response

Information

[ ] The official example scripts
[ ] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction

https://github.com/triton-inference-server/tensorrtllm_backend/

Followed the official guide with llama3-8b chat model

Expected behavior

Loads all the model Serves LLM with gRPC / https at 8000, 8001 ports

actual behavior

I0617 11:16:30.692886 31127 server.cc:307] Waiting for in-flight requests to complete. I0617 11:16:30.692893 31127 server.cc:323] Timeout 30: Found 0 model versions that have in-flight inferences I0617 11:16:30.692914 31127 server.cc:338] All models are stopped, unloading models I0617 11:16:30.692919 31127 server.cc:347] Timeout 30: Found 0 live models and 0 in-flight non-inference requests error: creating server: Internal - failed to load all models error: creating server: Internal - failed to load all models

Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted.

mpirun detected that one or more processes exited with non-zero status, thus causing the job to be terminated. The first process to do so was:

Process name: [[3135,1],1] Exit code: 1

additional notes

W0617 11:16:30.423990 31125 server.cc:251] failed to enable peer access for some device pairs [libprotobuf ERROR /tmp/tritonbuild/tritonserver/build/_deps/repo-third-party-build/grpc-repo/src/grpc/third_party/protobuf/src/google/protobuf/text_format.cc:335] Error parsing text-format inference.ModelConfig: 29:17: Expected integer, got: $ E0617 11:16:30.424716 31125 model_repository_manager.cc:1335] Poll failed for model directory 'tensorrt_llm': failed to read text proto from tensorrtllm_backend/triton_model_repo/tensorrt_llm/config.pbtxt I0617 11:16:30.424744 31125 server.cc:607]

Above in bolds seems to be the issue.

I don't know how to resolve above.

Please help.

triton-inference-server / tensorrtllm_backend