add prometheus metrics for Genie

Netflix / genie

Distributed Big Data Orchestration Service

https://netflix.github.io/genie

Apache License 2.0

1.7k stars 365 forks source link

add prometheus metrics for Genie #1203

Closed sbock-slack closed 3 weeks ago

sbock-slack commented 3 months ago

Add dependencies to Genie that are needed to support the /admin/prometheus endpoint.
We've built our own JARs and are already using Prometheus to power our metrics around Genie's health. Hoping to remove the need for us to locally fork Genie by getting this into open source 👍

babourine-slack commented 3 months ago

👋 Hi @bhou2 ! Thank you for hard work on maintaining Genie! Was wondering if we could get some traction on this PR? Will be happy to adjust it if needed and will appreciate an approx ETA so we could adjust accordingly.

bhou2 commented 2 months ago

Thanks @babourine-slack I did not get the notification about this PR in the past several weeks. Checking.

babourine-slack commented 2 months ago

Could you clarify the tests you have done?

We built it locally with all tests as well as running it in prod for weeks now w/o any issues. If there is any specific process that we should follow to contribute we'll be happy to do. There are two areas we would like to contribute in:

Modernize docker container that goes to the dockerhub (primarily because of the number of critical security issues baked into it)
Bump versions of some dependencies up as unfortunately the current jars carry many critical CVEs due to those dependencies. :(

babourine-slack commented 1 month ago

👋 Hi @bhou2 ! Hope all's well! Dont know if ^ fell thru for you in notifications as your message for me 🤦 (I had to set some :) ) Pls let me know if you need any additional context. Also, if it'll help, we can setup a meeting just to meet and greet the teams on both sides (zoom, goog meet, ....) -- let me know if that's something you'll be interested in.

sbock-slack commented 1 month ago

Hey @bhou2, nice to meet you. Freshly back from paternity leave, looks like @babourine-slack has already answered with all the details I have. Definitely echoing his comment, happy to hop on a meet/zoom if it would help build confidence in our changes 👍

bhou2 commented 3 weeks ago

Great meeting with your team. Looking forward to our continuous communications and collaborations. PR approved as you confirmed that this code change has been running and tested in production for a long time.

babourine-slack commented 3 weeks ago

Thanks @bhou2 ! It was great meeting you and the team! We're looking forward to working with you on the project!