Spectator Middleware (Question)

[bitbier]

We are currently using a Netflix OSS application that we want to be able to take their metrics that they are produce and ship them over to Datadog. We've recently tried using something like https://github.com/coursera/metrics-datadog to ship metrics generated/collected by spectator over to Datadog, but currently there are some weird implementation details with how the metrics are generated. For example, the metrics-datadog implementation ships everything as a gauge rather than using StatsD/Datadog support metrics like counters, histograms, etc. I think this is because this is how the metrics frameworks collects this information.

I recently stumbled upon this library via a comment https://github.com/coursera/metrics-datadog/issues/45#issuecomment-364797284.

We'd like to use micrometer library, but it seems like there isn't an adapter for Spectator -> Micrometer -> Datadog. Instead, this library seems like a replacement for Spectator. Unfortunately, we don't control the OSS application. It seems like there will have to be some kind of work on our end to create a Spectator -> Micrometer registry that we can then add a DatadogReporter or some kind to ship the metrics off.

We were wondering if there was any built-in way to create this adapter registry or if someone else has done this before, before going down this path.

[jkschneider]

It's a good question. Since Micrometer's meter types are heavily influenced by Spectator, such a mapping isn't difficult. Here is a most-of-the-way-there implementation. Just needs the implementations for Gauge, Timer, Distribution Summary, which are wholly similar to the Counter.

Something like this may make sense in spring-cloud-netflix.

cc / @spencergibb

[bitbier]

@jkschneider Thanks for the pointers. I went ahead and implemented the adapter for Spectator -> Micrometer. I'll be doing some testing (mostly just making sure we get the metrics that we want). If it at least works, I can create a PR to add it into some kind of adapter module if you are interested in such things.

Let me know where you want me to fit this code into this project and I can get on that.

Thanks again.

[jkschneider]

@bitbier I think it makes sense to issue such a PR to spring-cloud-netflix. Feel free to tag me there!

(BTW, are you using Netflix OSS via spring-cloud-netflix?)

[bitbier]

We are actually not even using Spring. We are running a conductor cluster and just wanted to get metrics shipped over to Datadog for monitoring the usage there.

Conductor has a way of loading guice modules for specific plugins and configurations, so we simply have a datadog metric module that we created to be able to ship those metrics out of the application.

[jkschneider]

Would still love to see the completed Spectator->Micrometer adapter. It may be that our friends at Netflix could take advantage of this as well.

[bitbier]

Yeah definitely. So considering that this doesn't really involve spring or rather it just provides the adapter base class. Does it make sense to get put into this repo as a sub project or would you still like me to push this to the spring-cloud-netflix guys or even I could try to push this over to the Spectator guys?

Either way, let me know where best to put this. I can also just publish this adapter in my own project so other can use it later. Just looking for a good place to park it.

[jkschneider]

Having not seen the implementation, my initial inclination is against having it a part of Micrometer core. I'd say classpath scanning as a "good idea" is a Spring perspective that is vigorously rejected elsewhere (but not everywhere). The hope was that each framework would adopt a model that is idiomatic to it. Absent any framework at all, I'm not sure what the right thing to do is.

[jonsie]

Here's an example of using Spectator with a Micrometer registry for those curious:

https://github.com/Netflix/spectator/tree/master/spectator-reg-micrometer

You can wire this up to report metrics to Datadog with either StatsdMeterRegistry or DatadogMeterRegistry.

[marcingrzejszczak]

Spectator has a meter registry in the Spectator's codebase. I guess that's good enough?

[github-actions[bot]]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

[github-actions[bot]]

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open.