Add GPU metrics to metrics logging

awslabs / multi-model-server

Multi Model Server is a tool for serving neural net models for inference

Apache License 2.0

998 stars 230 forks source link

Add GPU metrics to metrics logging #821

Open mikeobr opened 5 years ago

mikeobr commented 5 years ago

Currently MMS_METRICS that are logged include things like CPUUtilization and Memory usage. It would be great if these also included things like GPU usage and memory.

This would make it easier to track how well we are utilizing a server's GPU resources as well as diagnose out of memory issues.

Thanks!

vdantu commented 5 years ago

@mikeobr : Thanks for raising this issue. We will look into how to get functionality in.