IBM / text-generation-inference

IBM development fork of https://github.com/huggingface/text-generation-inference
Apache License 2.0
57 stars 30 forks source link

:recycle: move metrics into one file #97

Closed joerunde closed 5 months ago

joerunde commented 5 months ago

Motivation

This PR accomplishes two things:

Modifications

This refactors all usages of the metrics crate into a single file, so that changes to how it's used can be made in one place. This lets us easily duplicate all the counter metrics.

Result

No existing behavior should change, only new *_total counters should be added to the /metrics endpoint.

joerunde commented 5 months ago

Here's an example output of /metrics from running locally: metrics.txt

joerunde commented 5 months ago

TravisCI test run here: https://v3.travis.ibm.com/github/ai-foundation/fmaas-inference-server/builds/19285089 (using a build pushed from this branch)

Seems to work fine on the dev pod 🤞🤞🤞