triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
8.38k stars 1.49k forks source link

test: Test model_metrics config and document histogram buckets override #7752

Closed yinggeh closed 2 weeks ago

yinggeh commented 4 weeks ago

What does the PR do?

Tests new model_metrics message in pbtxt. Add example to override histogram buckets per-family.

Checklist

Commit Type:

Check the conventional commit type box here and add the label to the github PR.

Related PRs:

https://github.com/triton-inference-server/core/pull/405 https://github.com/triton-inference-server/common/pull/126

Where should the reviewer start?

Test plan:

L0_metrics

Caveats:

Background

Default histogram buckets does not satisfy all use cases.

yinggeh commented 2 weeks ago

Verified CI has completed without introducing new failures. @rmccorm4 @GuanLuo