Metrics

[mdelaurentis]

I've been adding experimental metrics logging to singer-python and a few taps. I'd like to add a best practice recommendation for logging metrics from Taps. I would really appreciate feedback on this best practice recommendation and the singer-python changes. Below are links to diffs for a few taps that use the new singer-python stats utilities, so you can see how the implementation would work in practice.

• Updated Best Practices guide: https://github.com/singer-io/getting-started/blob/metrics/BEST_PRACTICES.md (look at that rather than the diff)
• singer-python - https://github.com/singer-io/singer-python/pull/19
• tap-shippo - https://github.com/singer-io/tap-shippo/pull/3/files
• tap-facebook - https://github.com/singer-io/tap-facebook/pull/3/files
• tap-closeio - https://github.com/singer-io/tap-closeio/pull/6/files

I'm particularly interested in answering feedback in the following areas:

• What terminology should we use? "stats"? "metrics"?
• Are the field names clear?
• For the singer-python change, is the distinction between a Counter and a Timer clear enough?

[KAllan357]

Using "metrics" definitely makes more sense than "stats."

I think the document does a good job explaining each of the fields.

As for the distinction between Timers and Counters, I'm wondering if we should be deferring this functionality (and also the terminology) to a library. As an example, I've used Dropwizard in a few projects and found it to be an excellent library. A quick search yielded a Python compatible library here.

I think using this library would take 3 steps:

1. Add our own implementation of a Reporter. Something like a JSON + stdout Reporter. This could live in singer-python.
2. Each tap gets its own MetricsRegistry which is assigned our Reporter
3. Each tap then can add whichever metrics that are appropriate Counter, Histogram, Meter, Timer, Gauge, CallbackGauge, SimpleGauge.

I think a pattern like this would give us a few benefits:

• We end up owning less code and are using a well defined and widely understood metrics pattern.
• We also get a larger number of metric types for "free".
• Emitting metrics occurs in a separate thread and doesn't get in the way of the tap's real work.

What do you think?

[mdelaurentis]

Thanks very much for the feedback @KAllan357. I agree that "metrics" makes more sense than "stats". I'll change it in the best practices guide.

Regarding the use of pyformance, I think we should evaluate it on a Tap. I'd like to see if we can get the same output as singer-python, and then see how the code looks if we use it. If we can get similar output and it's easy to use, I'm open to it.