search

yabeda-rb / yabeda-prometheus

Adapter to expose metrics collected by Yabeda plugins to Prometheus
MIT License
113 stars 18 forks source link

Support Yabeda server in a new process #22

Open etsenake opened 2 years ago

etsenake commented 2 years ago

Should fix #21

I've been working on getting yabeda into our app which runs on passenger. I finally have it running, but faced a couple of issues that I've been able to fix manually in our app but wanted to upstream some of those changes Issue 1: Thread gets killed by passenger as a long-running request Issue 2 ✅ : Metrics become pretty bloated when we haven't deployed in a while I imagine this would be a problem on weekends where the app is up for days. The addition of rack deflater shrunk the size of the metrics logged by close to 90% in my testing: With Rack deflater Without Rack deflater
1355 15918
1538 16851
1582 17784

Depending on your scrape timeout on your prometheus server this could or could not be a problem ours is set pretty low but has worked fine for other exporters we have.

Issue 3[Just a minor annoyance 😅 ]: When running in clustered mode if app isn't preloaded the other processes or threads error out with Errno::EADDRINUSE doesn't kill the app just pollutes the start up logs a bit

Changes

Gearing up to go live Monday with these changes and a couple others others in relation to resque on Monday so any feedback would be greatly appreciated.

Shameless plug: because we use resque I wrote a small extension for metrics around activejob for us to get some insight similar to what y’all have done for sidekiq and faktory. Check it out here

etsenake commented 2 years ago

I could also split this up into multiple PRs if that makes it easier to review unsure who to add as a reviewer though @Envek are you the person to review? 😅

Envek commented 2 years ago

Also added your yabeda-activejob to the Yabeda plugin list. Thanks for sharing!

etsenake commented 2 years ago

Awesome! Apologies for taking so long to get back to this was at a work conference for the past couple days getting those changes in, test some things around calling configure! over the next little bit. I'll ping you once it's ready for another