search

datahub-project / datahub

The Metadata Platform for your Data Stack
https://datahubproject.io
Apache License 2.0
9.45k stars 2.8k forks source link

Expose operational metrics #2214

Closed pedro93 closed 3 years ago

pedro93 commented 3 years ago

Is your feature request related to a problem? Please describe. Once datahub is deployed, regardless of method (docker-compose, K8s, bare-bones, etc...) there is no way to obtain metrics & alerts related to the health of the deployed components. I.e: There is no endpoint I can poll to retrieve the health of GMS. Should it fail, I have no way to automate alerts to notify me.

Describe the solution you'd like Given that DataHub is a mostly java project it should be relatively easy to expose JMX metrics such that they can then be consumed by a Prometheus instance or similar. Ideally there should exist operational metrics for each component of Datahub with associated documentation on how to use it.

shakti-garg commented 3 years ago

Agree for need of exposing JMX/Prometheus(which is widely used nowadays) metrics. For MCE and MAE, it is pretty-much out-of-box using Spring actuator APIs. For GMS also, can also be done using JMX exporter.

In case you agree it to be relevant for datahub, happy to contribute to this feature.

pedro93 commented 3 years ago

I think it is very relevant but I'm not a maintainer or contributor to the overall project. I simply created this issue as a request from a slack thread in the official channel.