I am running 0.7.1triton server with TensorRT model.
I0429 12:46:50.778379 1 grpc_server.cc:2519] Started GRPCInferenceService at 0.0.0.0:8001
I0429 12:46:50.778524 1 http_server.cc:4623] Started HTTPService at 0.0.0.0:8000
I0429 12:46:50.819315 1 http_server.cc:315] Started Metrics Service at 0.0.0.0:8002
Logs are getting generated properly when I am making the HTTP call, however that is not the case for GRPC (Langchain-nvidia-trt). Anything I need to do differently? Please suggest.
Second concern:
In case of multiple threads, I am sometimes getting the error below. Is there any configuration specific for GRPC?
[StatusCode.UNAVAILABLE] failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:8081: Failed to connect to remote host: Connection refused
Hi team,
I am running
0.7.1
triton server with TensorRT model.I have configured logging:
Logs are getting generated properly when I am making the HTTP call, however that is not the case for GRPC (Langchain-nvidia-trt). Anything I need to do differently? Please suggest.
Second concern:
In case of multiple threads, I am sometimes getting the error below. Is there any configuration specific for GRPC?
[StatusCode.UNAVAILABLE] failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:8081: Failed to connect to remote host: Connection refused
Regards Tapan