Attach stack trace to events from `HTTPClient` integration so they are not merged (fingerprinted) into a single issue

[harry-gocity]

Is there an existing issue for this?

• [X] I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• [X] I have reviewed the documentation https://docs.sentry.io/
• [X] I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/nextjs

SDK Version

7.53.1

Framework Version

React 17.0.2, Next.js 12.1.6

Link to Sentry event

https://go-city.sentry.io/issues/4231630878/?project=4503958434480128

SDK Setup

const { publicRuntimeConfig } = getConfig();

const isProduction = process.env.NODE_ENV === 'production';

Sentry.init({
    // when dsn is null, no errors will be sent to Sentry
    dsn: isProduction ? publicRuntimeConfig.SENTRY_DSN : null,
    environment: publicRuntimeConfig.ENVIRONMENT ?? 'local',
    release: isProduction ? publicRuntimeConfig.SENTRY_RELEASE : `development-${publicRuntimeConfig.USER}`,
    ignoreErrors: ['jQuery'],
    allowUrls: isProduction
        ? [
              // Match our app 
              /https://our-site-here.com/,
              // Match gtm error
              /gtm.js/,
          ]
        : [],
    tracesSampler,
    beforeSend: (event: Event): Event => {
        // append LogRocket Session url to Sentry extras, for easier debugging
        const logRocketSession = LogRocket.sessionURL;
        if (logRocketSession !== null) {
            event.extra['LogRocket'] = logRocketSession;
        }
        return event;
    },
    integrations: [
        new HttpClient(),
    ],
});

Steps to Reproduce

We added the HTTP Client Integration to our Sentry setup:

    integrations: [
        new HttpClient(),
    ],

After this, we started to see events being tracked in Sentry for 500, 502 and 504 status codes in responses from our site.

Expected Result

We would be able to triage requests to different URLS separately. Sentry would group HTTPClient-captured events as separate issues. We thought that would happen by default using the request url and response status code.

Actual Result

All the events are grouped under the same Sentry issue, regardless of the error code or request URL.

For the issue linked above:

Under the 'all events' tab there several different issues all being grouped together (URLs cropped intentionally):

AFAIK we have not accidentally merged any issues manually.

[lforst]

Thanks for writing in! This is a very valid point. We'll try to fix this.

[harry-gocity]

Hi @lforst any update on this?

[lforst]

@harry-gocity nope you would see it in this issue!

[harry-gocity]

Hi @lforst just wondering if you could give any estimates as to if/when this would hit roadmap for a Sentry release. We currently have 8.4k events all grouped in Sentry under the same HTTP Client Error with status code: 500 issue. This is completely unmanageable and untriageable for us.

I'm currently upgrading us from 7.77 to 7.99 and saw that HttpClient is deprecated. Are there any changes in the httpClientIntegration that can help us? Otherwise, should we be separating these via a custom beforeSend and applying a fingerprint ourselves? Is there any other recommended approach to handling this?

[lforst]

@harry-gocity There are no real changes besides the renaming of it. We are currently not working on this issue and likely won't do so for the coming month. I cannot give a timeline.

We do not have any specific recommendations for grouping. Adding custom grouping in beforeSend seems fine to me.

[bryanjtc]

Is there any solution for this or an eta for a fix?

[lforst]

@bryanjtc No, this feature just needs some love. It's not really a bug, but the created errors need proper stacktraces and grouping.

[bryanjtc]

@bryanjtc No, this feature just needs some love. It's not really a bug, but the created errors need proper stacktraces and grouping.

Do you have an example for a better grouping strategy using beforeSend?

[lforst]

@bryanjtc I guess you could look at the request context and looks at the url, method, headers, and cookies value and group however you like.

https://github.com/getsentry/sentry-javascript/blob/08a3b7c6ec9bd4d19ebe49ecfaca11a7c997e356/packages/browser/src/integrations/httpclient.ts#L360-L384

[alexmelagrano]

For anyone else who finds this -- transforming the default ErrorEvent objects was quite easy, and is working well for us.

Here's a simplified version of what I set up using beforeSend:

beforeSend(event, hint) {
  ...

  // Naively looking for events with the default `HTTP Client Error with status code: 500` message
  if (event.message.match(HTTP_SENTRY_ERROR_REGEX) {
    if (event.contexts?.response && event.request) {
      const requestStatusCode = errorEvent.contexts.response.status_code;
      const requestMethod = errorEvent.request.method;
      const requestUrlPath = errorEvent.request.url ? new URL(errorEvent.request.url).pathname : 'unknown';
      // Util for scrubbing path IDs
      const cleanedRequestUrlPath = cleanUrlPath(requestUrlPath);

      const newErrorMessage = `HTTP Client Error: ${requestMethod} ${cleanedRequestUrlPath} returned with code ${requestStatusCode}`;

      // Override the displayed message
      event.message = newErrorMessage;

      // Override the actual exception value; this is how Events are categorized into Issues
      if (event.exception?.values && event.exception?.values[0].value?.match(HTTP_SENTRY_ERROR_REGEX)) {
        event.exception.values[0].value = newErrorMessage;
      }
    }
  }

  ...

  return event;
}

Error message string is obviously up to you, but I figured the method/path/code were the important bits I wanted these Issues to group themselves by. Below is a screenshot of our React project's related issues before & after we deployed this V1 solution; you can see the generic error at the top, then the more detailed errors splitting out from that.

I'll also note that we're running this solution in our React Native app, and the solution was identical. The only annoyance is that the types for beforeSend don't line up by default, despite having all the same information.