Deployement failed for BERT

[vivekjoshi556]

I have a bert model that I am trying to deploy with Triton Inference Server using Tensorrt-LLM backend. But I am getting errors:

? Docker Image: 24.03 ? TensorRT-LLM: v0.8.0

Error: +-------+---------+-----------------------------------------------------------------------------------------------------------------------+ | Model | Version | Status | +-------+---------+-----------------------------------------------------------------------------------------------------------------------+ | bert | 1 | UNAVAILABLE: Internal: unexpected error when creating modelInstanceState: [json.exception.out_ofrange.403] key 'num | | | | layers' not found | +-------+---------+-----------------------------------------------------------------------------------------------------------------------+

I followed the guide exactly but don't know if the problem is with tensorrt llm or the backend.

[byshiue]

Could you share what guide do you follow?