search

stripe / veneur

A distributed, fault-tolerant pipeline for observability data
MIT License
1.73k stars 174 forks source link

Solutions for Tee-ing statsd traffic #554

Open SleepyBrett opened 5 years ago

SleepyBrett commented 5 years ago

I have a weird edgecase scenario and can't tell if veneur can support it. I'd like to tee off some statsd traffic that is sent to a veneur instance. I'd like to send it to both datadog and a local prometheus (I assumed via the statsd exporter). But I don't see a sink for statsd or prometheus. Any advice?

cory-stripe commented 5 years ago

Interesting! No, we don't have either of those as sinks, but a (Dog)StatsD sink would be kinda cool. Would turn it into a proxy!

SleepyBrett commented 5 years ago

yeah i was thinking 

clients -> veneur -> datadog
              +-----> prometheus-statsd-exporter

The why's and wherefores:

Datadog doesn't have a kubernetes metrics bridge, but we have several client-teams that would like to autoscale on custom metrics. Since DD is our corporate standard, and since internally in the kube team we use prometheus to monitor our clusters, and since the most stable metrics bridge is for prometheus we were looking for a way to allow them to scale using custom metrics but not force them to double export their metrics.

cory-stripe commented 5 years ago

Most of our internal work is aligned around tracing atm, so I don't think anyone on the team would get around to poking at this for a while. If you're interested in throwing together a DogStatsD sink we'd certainly be happy to review and merge it tho. 😄

SleepyBrett commented 5 years ago

That's our plan! Just trying to lay the groundwork (and make sure I didn't miss anything) for the pull request.