open-telemetry / opentelemetry-dotnet-contrib

This repository contains set of components extending functionality of the OpenTelemetry .NET SDK. Instrumentation libraries, exporters, and other components can find their home here.
https://opentelemetry.io
Apache License 2.0
449 stars 273 forks source link

Hangfire jobs not considered transactions #489

Open AlexandruRus23 opened 2 years ago

AlexandruRus23 commented 2 years ago

Issue with OpenTelemetry.Instrumentation.Hangfire

List of all OpenTelemetry NuGet packages and version that you are using (e.g. OpenTelemetry 1.0.2):

OpenTelemetry.Instrumentation.Hangfire 1.0.0-beta.1 OpenTelemetry.Extensions.Hosting 1.0.0-rc8

Runtime version (e.g. net462, net48, netcoreapp3.1, net6.0 etc. You can find this information from the *.csproj file):

net6.0

Is this a feature request or a bug?

What is the expected behavior?

I expected to see the hangfire jobs listed as transactions.

What is the actual behavior?

Hangfire jobs are not listed as transactions and will not generate other metrics or insights.

Additional Context

I'm using New Relic to view the telemetry for a .net6 application that only runs hangfire jobs. Because the Activities are "Internal", they are not listed as transactions. It seems like the conditions for traces to appear there is to have them as 'server' or 'consumer'. https://docs.newrelic.com/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/view-your-data/opentelemetry-transactions-page

I'm aware that this could be resolved by a custom implementation, but was wondering if it's worth raising this issue. I'm not familiar with the exact intent of each activity type and probably different APM services have different interpretations about this.

alanwest commented 2 years ago

@AlexandruRus23 👋 I work for New Relic.

The concept of a "transaction" is a New Relic specific thing. Logically, you can think of it as the portion of a trace scoped to a single service participating within the trace. Admittedly, it does not have a perfect mapping to any concept in OpenTelemetry.

For HTTP services, at least, the mapping is relatively clean - that is, the OpenTelemetry HTTP semantic conventions declare:

For an HTTP server span, SpanKind MUST be Server.

So, New Relic uses this to identify the "entry point" span created by each service within the trace. We do a similar thing for gRPC services as well as messaging systems where span kind=consumer is considered the entry point.

However, OpenTelemetry currently has no such conventions New Relic can use to identify a span as an entry point for background jobs started by frameworks like Hangfire.

I can't say whether it makes sense for initial spans created by background jobs to be server, consumer, internal, or something else. This needs to be a discussion at the spec level. Check out https://github.com/open-telemetry/opentelemetry-specification/issues/1347.