System metrics are not shown on port 8082 & Making inference API results as metrics

[yunjjun]

Hello, If you look at this page, torchserve provides System metrics. (https://pytorch.org/serve/metrics.html)

However, when torchserve is included in prometheus and looked up, System metrics are not shown. I wonder if there is a need to change the setting or if there is a special method. (https://pytorch.org/serve/metrics_api.html)

Additionally, I am wondering if it is possible to make the value of the inference API in the form of metrics and send it to prometheus.

Thank you for your reply.

[harshbafna]

@yunjjun: Metrics logging and metrics API are two different things in TorchServe :-)

Please refer following documentation for metrics_api and integration with Prometheus:

https://pytorch.org/serve/metrics_api.html

[yunjjun]

Thanks for answering! I have two questions about your answer. └ 1. Why did you do that? └ 2. Are there any workarounds or plans to expose Metrics (logging) to Metrics API?

And I think you were confused by asking two questions at once. Another question is how to get metrics from #1 like #2 from Prometheus. Please let me know about it :-)

[#1]

[#2]

[harshbafna]

@yunjjun: The current matrics API does not return System Matrix data and is tightly coupled to Prometheus and returns the data in Prometheus specific format.

There is a work in progress (#722) where :

• Metrics API will return all the metrics data in fix format including the system metrics as well.
• Prometheus metrics collector will be converted to a plugin that will consume the data from metric API and format it to Prometheus specific format.
• This is will also allow the user to plugin their own metric collection framework and apply custom formating.

This enhancement will roll out with the next release.

[yunjjun]

Thank you so much for answering ! 👍 I'm looking forward to seeing the next release soon :)

[hungtooc]

its 2022 and system metrics not added to API metrics yet