search

triton-inference-server / fil_backend

FIL backend for the Triton Inference Server
Apache License 2.0
68 stars 35 forks source link

Server response stats not populated with perf_analyzer #106

Closed mradzihovsky closed 3 years ago

mradzihovsky commented 3 years ago

When using perf_analyzer with fil_backend, server response stats are not populated.

For example, here is part of the output for a simple xgboost model where the Server Request count is 0:

Client: 
    Request count: 10969
    Throughput: 2193.8 infer/sec
    Avg latency: 448 usec (standard deviation 17 usec)
    p50 latency: 444 usec
    p90 latency: 471 usec
    p95 latency: 481 usec
    p99 latency: 513 usec
    Avg HTTP time: 441 usec (send/recv 69 usec + response wait 372 usec)
  Server: 
    Request count: 0