Filtering healthcheck transactions with sentry-opentelemetry-agent V8

[jebbench]

Problem Statement

I want to filter out my healthcheck transactions (grpc.health.v1.Health/Check, PING), preferably configured using an Environment variable.

I am using the new version of the sentry-opentelemetry-agent which works great, my only issue is filtering out healthcheck transactions.

Solution Brainstorm

In a perfect world I could provide an environment variable with a list of transactions to exclude (SENTRY_EXCLUDE_TRANSACTIONS=grpc.health.v1.Health/Check,PING) but I would settle for being able to configure it in code while still being able to take advantage of the auto initialisation (I've tried calling Sentry.init and just specifying a beforeSendTransaction options but I get the error "DSN is required. Use empty string or set enabled to false in SentryOptions to disable SDK.").

[adinauer]

Thanks for the feedback, happy you like the agent!

We'll take a look at options to solve this and report back here.

[adinauer]

@jebbench have you taken a look at https://docs.sentry.io/concepts/data-management/filtering/#transactions-coming-from-health-check by any chance? It could help reduce the issue until we implement something to filter them out in the SDK and not send them at all.

[jebbench]

@adinauer I have and we've got the filter enabled, unfortunatly the patterns are all based on HTTP requests (they work great for filtering our /healthz eps) but they don't match GRPC or Redis requests (grpc.health.v1.Health/Check and PING).

I did send feedback in the app to ask about adding the GRPC service (as it's a standard GRPC service used by Kubernetes and the likes) but I've not heard back.

[adinauer]

@jebbench I've relayed this internally and hope that someone will take a look at your feedback regarding filtering GRPC health checks in product / relay soon.

[adinauer]

We're planning to implement a filter very similar to what you suggested. There's also an existing issue (https://github.com/getsentry/sentry-java/issues/2660) and JS SDK also has something similar (https://docs.sentry.io/platforms/javascript/guides/express/configuration/filtering/#filtering-transaction-events).

Our goal is to have this implemented and released as part of another v8 beta soon.

[adinauer]

We have just released 8.0.0-beta.2 of the Java SDK, which allows you to configure transactions you want to ignore based on their name. There's some examples in the description of https://github.com/getsentry/sentry-java/pull/3871 . If you decide to give it a try, feedback would be great.

[jebbench]

We've tested this over the past couple of days and it works really nicely, thanks for getting it done and release so quickly!

[adinauer]

Thanks for the feedback!