search

elastic / apm

Elastic Application Performance Monitoring - resources and general issue tracking for Elastic APM.
https://www.elastic.co/apm
Apache License 2.0
383 stars 114 forks source link

Elixir support #60

Open JCMais opened 5 years ago

JCMais commented 5 years ago

It would be really great if we could support Elixir with an official APM client.

From what I could see new clients is not a high priority right now, but I'm opening this issue to track related projects or non-official libraries.

The only APM related client I could see for APM was that one, which can be useful to get inspiration from: https://github.com/scoutapp/scout_apm_elixir

marceldegraaf commented 5 years ago

The folks over at the Spandex project have an adapter based tracing solution for Elixir/Phoenix apps. Currently there is only one adapter, shipping traces to Datadog, but it should be possible to build a generic OpenTracing adapter that is usable with Elastic APM.

marceldegraaf commented 5 years ago

Update: I'm working on an Elastic APM agent for Elixir. The basic implementation works, and is able to track transactions and related spans and errors:

image

image

This is all very much work in progress, but I'm pretty sure I'll be able to ship a useful first version in the coming 2-3 weeks. I'll link to the repository here once it's published.

JCMais commented 5 years ago

@marceldegraaf this looks really amazing, thanks for taking the time to do that.

marceldegraaf commented 5 years ago

Update: working towards the first stable-ish release of the APM agent for Elixir. Will report back here once the first release is shipped.

mikker commented 5 years ago

Hi there! I'm the (currently only) developer of the official Elastic APM Ruby agent.

I spent a week on an Elixir agent a few months ago as an experiment but didn't get far enough that I wanted to share it. Not an Elixir expert but I love the language. I'll gladly review PRs and supply knowledge on the details of Elastic APM if you'd like. Anything I can to help with this, let me know.

Also happy to send some swag your way for your efforts if you're into that sort of thing 😉

marceldegraaf commented 5 years ago

@mikker awesome, is it OK if I ping you before the first public release to review if I made the right assumptions about how the Elastic APM data model and API's work?

mikker commented 5 years ago

Of course it is! Just ping me and I'll give it a look 👍 If you have any feedback on the process we'd also very much like to hear it.

marceldegraaf commented 5 years ago

FYI: still working on this. It's all taking longer than expected, as usual with these things 😉. I'll update here when I've got something to share!

tsloughter commented 5 years ago

If Elastic adopts OpenTelemetry (the merger of OpenCensus and OpenTracing) there will be Erlang and Elixir support later this year -- currently there is OpenCensus support and OpenTelemetry planned.

The Erlang Foundation's Observability Working Group is working to ensure support for APMs across the BEAM.

pranayhere commented 5 years ago

@marceldegraaf are you still working on elixir-apm?

tsloughter commented 5 years ago

I should have also mentioned that the Spandex team is planning to sunset their project and move people to OpenTelemetry. So if the plan is to do a Spandex plugin for Elastic APM you might want to consider OpenTelemetry instead.

marceldegraaf commented 5 years ago

My Elixir APM agent is set up as a standalone thing, not a Spandex plugin. @pranayhere still working on it but haven't found the time to cut a release. Will try to do that soon!

tsloughter commented 5 years ago

OpenTracing is also being "replaced" by OpenTelemetry. So I assume at some point Elastic will have to support it as well but I don't know if they are currently working on it or involved.

If anyone from Elastic sees this maybe they can give an update on the plans there, since there will be an Elixir client if/when they make that transition.

mikker commented 5 years ago

We support OpenTracing in all our official agents. OpenTracing(/Telemetry) is mostly just an API for tracing so someone would still have to write the tracer part of the agent that sends the data to APM Server. All the vendors have their specific way to report and keep the data so don't expect any OpenTelemetry library to just automagically work with all vendors, including Elastic APM.

We still don't have any official plans to build an Elixir agent, however I'll gladly help anyone up to the task as much as I can.

tsloughter commented 5 years ago

OpenTelemetry is not just an API. It does compromise between OpenCensus and OpenTracing by splitting the API and implementation, but unlike OpenTracing it does not leave almost everything up to interpretation and implementation.

Hopefully to be able to say a service/vendor "supports OpenTelemetry" will require supporting the SDK (the default implementation of the API that is available for each language) and not simply "implement the API" -- see https://github.com/open-telemetry/opentelemetry-specification/issues/185. This has been the biggest issues with OpenTracing for languages that aren't officially supported by vendors and why I'm pushing this to be included in the OpenTelemetry spec.

If vendors would support the SDK then any language that has an implementation of the SDK, like Elixir and Erlang will, is supported, even if it is limited in features compared to the vendor's SDK.

GregMefford commented 4 years ago

Hey there! I'm a co-maintainer on the Spandex project and I just wanted to chime in and agree with what Tristan said about our intent to migrate users to the OpenTelemetry project and sunset Spandex at some point in the future. Given that OpenTelemetry as a project is intended to be a convergence of both OpenTracing and OpenCensus, that's where future effort should be directed. We plan to do what we can to provide migration guides and as much interoperability as we can, but the Spandex project itself will not see much active development going forward.

tsloughter commented 4 years ago

OpenTelemetry is now officially Beta for a number of languages including Erlang/Elixir https://medium.com/opentelemetry/opentelemetry-is-officially-in-beta-352fa859db10

Integrations and exporters are being published to hex under the opentelemetry org https://hex.pm/orgs/opentelemetry

There is someone working on Elastic APM exporter for the Elixir client but it isn't clear cut because of the differing data models.

And I wanted to stress again that OpenTelemetry is not just an API, meaning the tracer itself is already implemented for the languages supported. And many vendors are working together on this so the data model will work across them.

austinlparker commented 4 years ago

OpenTelemetry is a complete API and SDK for tracing and metrics -- speaking for the project, we'd be extremely happy if there was an officially supported telemetry exporter for Elastic APM from Elixir (and the rest of the OpenTelemetry SDKs, for that matter!).

tsloughter commented 4 years ago

@mikker

I was asked about this again today in the Erlang community so wanted to comment again to make clear that OpenTelemetry is not just an API but is also a protocol and a collector/agent.

Elastic could be added to the Otel collector as an exporter https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/master/exporter like honeycomb, lightstep, stackdriver, etc and allow any language with an OpenTelemetry implementation to report their spans to Elastic APM.

graphaelli commented 4 years ago

@tsloughter We're really glad to hear there is interest from the Erlang community and do intend to implement an exporter for the collector - we're tracking that in https://github.com/elastic/apm/issues/98.

tsloughter commented 4 years ago

Great to hear!

Purvansh-gourh commented 4 years ago

@marceldegraaf any info on elixir-apm you were working on?

marceldegraaf commented 4 years ago

Nope, no progress unfortunately. But it seems the best way forward is to look into OpenTelemetry, as mentioned above.

graphaelli commented 4 years ago

In case you weren't also following #98, that work has been merged and getting started docs are up - looking forward to your feedback.

jnatherley commented 4 years ago

https://www.elastic.co/blog/elastic-apm-opentelemetry-integration