Add Triton Cache metrics

triton-inference-server / local_cache

Implementation of a local in-memory cache for Triton Inference Server's TRITONCACHE API

BSD 3-Clause "New" or "Revised" License

5 stars 1 forks source link

Closed rmccorm4 closed 1 year ago

rmccorm4 commented 1 year ago

[x] check interaction with Triton when --allow-metrics=false, and when Triton is built with -D TRITON_ENABLE_METRICS=off.