search

mozilla-services / autopush

Python Web Push Server used by Mozilla
https://autopush.readthedocs.io/
Mozilla Public License 2.0
215 stars 34 forks source link

Add metric handler for grafana #1400

Closed jrconlin closed 4 years ago

jrconlin commented 4 years ago

Currently tags are getting dropped after moving from Datadog. We are now defaulting to TwistedMetrics, which ignores tags.

Grafana tags should be specified as key=val

pjenvey commented 4 years ago

The grafana/influxdb metrics collector (telegraf) prefers key=val but our key:val format seems to work with it, given the example of autopush-rs emitting key:val via cadence.

Problem here is the underlying statsd client of TwistedMetrics (txtstatsd) does not support metrics tags.

sciurus commented 4 years ago

Telegraf supports dogstatsd protocol, so you should be able to revert back to however your statsd client was configured before.

https://github.com/mozilla-services/cloudops-docs/blob/master/Infrastructure/Monitoring/Monitoring%20Guidance.md#collect-high-volume-application-metrics are the current recommendations

jrconlin commented 4 years ago

The code uses a dedicated DatadogMetrics library, so guessing that it's more than just tweaking a config to use a crap API key. Instead I just wired in the markus library.

pjenvey commented 4 years ago

Oops: I thought the existing dogstatsd backend we use would work w/ statsd, apparently it doesn't.