Sentry strips Fastify http.route attribute with OpenTelemetry on metrics

nathanscully commented 1 day ago

Is there an existing issue for this?

[x] I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
[x] I have reviewed the documentation https://docs.sentry.io/
[x] I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/node

SDK Version

8.36.0

Framework Version

Fastify 4.28.0

Link to Sentry event

No response

Reproduction Example/SDK Setup

https://github.com/nathanscully/otel-sentry-fastify

Repo contains three different versions of Fastify / Sentry / Otel setup

Steps to Reproduce

Clone the repo and run the various start commands against an otel-collector.

Only OTEL

pnpm start:otel will run a standard otel setup, which patches Fastify and emits metrics. In the otel collector you can see the http.route metric includes:

Metric #0
Descriptor:
     -> Name: http.server.duration
     -> Description: Measures the duration of inbound HTTP requests.
     -> Unit: ms
     -> DataType: Histogram
     -> AggregationTemporality: Cumulative
HistogramDataPoints #0
Data point attributes:
     -> http.scheme: Str(http)
     -> http.method: Str(GET)
     -> net.host.name: Str(localhost)
     -> http.flavor: Str(1.1)
     -> http.status_code: Int(200)
     -> net.host.port: Int(3000)
     -> http.route: Str(/) <----- IMPORTANT
StartTimestamp: 2024-11-04 22:10:03.643 +0000 UTC
Timestamp: 2024-11-04 22:11:24.65 +0000 UTC
Count: 2060

Sentry Docs Example

pnpm start:sentry-docs will run the copy/paste from the Otel Sentry docs. I can't get any metrics to emit to Otel doing it this way. Would love an example if I am doing something wrong here. Note that when I run this I still get the following errors and warnings:


Sentry Logger [log]: Initializing Sentry: process: 60425, thread: worker-2.
Sentry Logger [error]: You have to set up the SentryPropagator. Without this, the OpenTelemetry & Sentry integration will not work properly.
Sentry Logger [log]: Integration installed: ...
...
..
Sentry Logger [log]: Running in ESM mode.
Sentry Logger [error]: You have to set up the SentrySpanProcessor. Without this, the OpenTelemetry & Sentry integration will not work properly.
Sentry Logger [warn]: You have to set up the SentrySampler. Without this, the OpenTelemetry & Sentry integration may still work, but sample rates set for the Sentry SDK will not be respected. If you use a custom sampler, make sure to use `wrapSamplingDecision`.
[09:18:32.930] INFO (60425): Server listening at http://0.0.0.0:3000

Which I find strange given that this is straight out of the docs - would be good to understand why its still throwing warnings and errors.

Sentry + Custom OTEL

pnpm start:otel-sentry will run my attempt at getting Sentry to play nice with Otel's NodeSDK. Traces emit ok, but the metrics from fastify have the http.route stripped, which is breaking all our reporting:

Metric #0
Descriptor:
     -> Name: http.server.duration
     -> Description: Measures the duration of inbound HTTP requests.
     -> Unit: ms
     -> DataType: Histogram
     -> AggregationTemporality: Cumulative
HistogramDataPoints #0
Data point attributes:
     -> http.scheme: Str(http)
     -> http.method: Str(GET)
     -> net.host.name: Str(localhost)
     -> http.flavor: Str(1.1)
     -> http.status_code: Int(200)
     -> net.host.port: Int(3000)
                       <--- IMPORTANT no http.route attribute present
StartTimestamp: 2024-11-04 22:38:01.18 +0000 UTC
Timestamp: 2024-11-04 22:38:48.472 +0000 UTC

Expected Result

Goal

Emit traces and metrics to a custom otel collector and use Sentry for error handling. If we also need to emit traces to Sentry to make this work, thats fine, but it's my preference not to.

What is not working

When using Sentry + Fastify + a custom Otel setup, Sentry appears to strip the http.route from the http.server.duration metrics making reporting by endpoint impossible.

Actual Result

Metric #0
Descriptor:
     -> Name: http.server.duration
     -> Description: Measures the duration of inbound HTTP requests.
     -> Unit: ms
     -> DataType: Histogram
     -> AggregationTemporality: Cumulative
HistogramDataPoints #0
Data point attributes:
     -> http.scheme: Str(http)
     -> http.method: Str(GET)
     -> net.host.name: Str(localhost)
     -> http.flavor: Str(1.1)
     -> http.status_code: Int(200)
     -> net.host.port: Int(3000)
                       <--- IMPORTANT no http.route attribute present
StartTimestamp: 2024-11-04 22:38:01.18 +0000 UTC
Timestamp: 2024-11-04 22:38:48.472 +0000 UTC

andreiborza commented 19 hours ago

Hi @nathanscully, thanks for writing in and providing comprehensive reproduction.

We'll take a look.

andreiborza commented 12 hours ago

@nathanscully could you try updating to 8.37.1 for both @sentry/node and @sentry/opentelemetry and try again? When I try with the latest version I don't get those errors anymore and I see http.route set correctly (looking at Sentry ui at least).

I tried looking through our fastify integration but we don't really strip the http.route anywhere. I'll have to have a deeper look.

getsentry / sentry-javascript