miketheman / heroku-buildpack-datadog

Heroku Buildpack to run Datadog DogStatsD in a Dyno
http://miketheman.github.io/heroku-buildpack-datadog
MIT License
55 stars 35 forks source link

Is integration with Heroku add-ons possible to collect data from them? #19

Open imochurad opened 8 years ago

imochurad commented 8 years ago

I have a java Heroku app that is utilizing a bunch of other services (Heroku add-ons). Like Redis, Cassandra, Kafka to name a few and I wanted to use Data Dog to collect health metrics and configure alerts if there are signs that my app becomes unhealthy. With your build pack, is that possible to configure integrations with the mentioned above add-ons? If so, could you please provide some details and/or examples? Reply would be appreciated.

miketheman commented 8 years ago

Hello @imochurad !

As can be seen in the startup script we run a specialized version of one of the Agent's components - the statsd listener - to allow apps to emit statsd messages to be sent to the platform.

This approach doesn't run the Agent as a collector for other sources like external databases - this would likely involve a more complex configuration and process runner approach. This might be something valuable for a standalone Dyno, not so applicable to a buildpack that is part of the application layer.

tatey commented 8 years ago

An alternative approach would be to setup a log drain for your app that parses runtime-metric logs and forwards them to Datadog.

miketheman commented 8 years ago

@tatey Cool idea! I don't see if the runtime-metrics provide insight into the add-ons, can you share how that's done?

tatey commented 8 years ago

If you're using the standard or premium tiers of Postgres and Redis you should see log entries for these addons in your app logs. I am not sure about Kafka, but given it's now a first class on Heroku, I imagine it's similar.

tatey commented 8 years ago

I actually open sourced an implementation of this idea last week in Ruby. It uses this very buildpack, so thanks!

imochurad commented 8 years ago

@tatey how do you parse a log drain?

tatey commented 8 years ago

@imochurad Heroku have a pretty good article on how Log Drains work.