search

getsentry / sentry-elixir

The official Elixir SDK for Sentry (sentry.io)
https://sentry.io
MIT License
627 stars 185 forks source link

[elixir] Performance powered by OTel #538

Open sl0thentr0py opened 1 year ago

sl0thentr0py commented 1 year ago

@krainboltgreene so there are several things that would need to be done to get performance tracing working with sentry-elixir. This SDK was so far handled by the community so it is not on par with what we call our Unified API for making common abstractions across all Sentry SDKs.

I will make 2 lists below, one of the bare minimum that would lead to quick opentelemetry based performance and the other more 'ideal' list where we make the SDK feature compatible in terms of performance with other SDKs.

Required

Optional

I will make a new issue out of this to track it in sentry-elixir. Unfortunately, this is a non-trivial amount of development work, so I can't give you very clear cut instructions on how you can contribute, but feel free to try stuff out and make a PR if you're interested and we can collaborate.

If there is sufficient interest from the community, we can also potentially prioritize me working on this as well next quarter.

Originally posted by @sl0thentr0py in https://github.com/getsentry/sentry/discussions/40712#discussioncomment-5268072

## Other notes
- [ ] @smeubank Make sure users understand how to upgrade to AM3 plans, in order to unlock this functionality. Maybe a note in the docs can already be enough and point users in the right direction on how to self-serve
tsloughter commented 1 year ago

What does "Hub" for concurrency in Elixir look like? It looks to be a context store? Like how OpenTelemetry we use the process dictionary?

Does this mean Sentry does no intend to adopt the OpenTelemetry API?

Also, what is "opentelemetry based performance" mean here? Based on what I read of the SpanProcessor model it doesn't read like you'd be relying on the Otel SDK for span operations, so if you found performance better with OpenTelemetry I wouldn't be sure thats the case when combined with the SpanProcessor. But I don't yet fully understand what it is doing, so I may be wrong there :)

sl0thentr0py commented 1 year ago

What does "Hub" for concurrency in Elixir look like? It looks to be a context store? Like how OpenTelemetry we use the process dictionary?

something like that yea, basically the Hub needs to be cloned per concurrency unit, so we'll need to spec out what that looks like for elixir.

Does this mean Sentry does no intend to adopt the OpenTelemetry API?

not directly, we have our own Tracing model and Ingestion so we will only support OpenTelemetry indirectly via the SpanProcessor pathway.

it doesn't read like you'd be relying on the Otel SDK for span operations

we'd rely on the Otel SDK for instrumenting and recording spans but we need to convert them to the Sentry model to be able to ingest and store them on our side, this is what the SpanProcessor would do.

cc @smeubank for high-level product design discussion ^

tsloughter commented 1 year ago

Now I see, so the SpanProcessor updates a global store of SpanId->SentrySpan and then OnEnd will update that based on the finished OpenTelemetry Span.

thbar commented 1 year ago

If there is sufficient interest from the community, we can also potentially prioritize me working on this as well next quarter.

I'm not sure how to provide feedback on that, but we (at https://transport.data.gouv.fr) would be very interested to see performance monitoring supported for Elixir.

josevalim commented 11 months ago

For what is worth, the Ruby one seems to be done via OpenTelemetry: https://docs.sentry.io/platforms/ruby/performance/instrumentation/opentelemetry/ ?

tsloughter commented 11 months ago

@josevalim I think it has both. And the OpenTelemetry option is awkward to implement -- uses a span processor to basically do a parallel tracking of spans. I don't know that there will be another option other than the processor though as long as Sentry requires the implementation to create Transactions.

hkrutzer commented 11 months ago

There is also a JS implementation using OpenTelemetry, indeed using a span processor.

sl0thentr0py commented 11 months ago

as long as Sentry requires the implementation to create Transactions.

@tsloughter @josevalim we have an ongoing project to move away from our Transaction model gradually on the ingestion side, will keep this thread updated when we ship something production ready.

jwaldrip commented 10 months ago

Any update on this?

whatyouhide commented 10 months ago

@jwaldrip no, and we'll post updates if there are any, no worries!

sl0thentr0py commented 10 months ago

I will actually start writing a spec for it this week!

sl0thentr0py commented 10 months ago

oki, current status of sentry ingestion of otlp traces follows!

Spec

Business concerns

Elixir SDK implications

I will start playing around with OpenTelemetry SDKs and exporters this week and update once ingestion works end-end.

sl0thentr0py commented 9 months ago

update: I have now added protobuf ingestion support because the elixir/erlang opentelemetry exporters only had http/protobuf support and not json. https://github.com/getsentry/relay/pull/3044

I will now test ingestion since this is in production.

whatyouhide commented 9 months ago

@sl0thentr0py lol this is fantastic news, I had already started working on the JSON export support 😄