Better document how to gracefully shutdown on SIGTERM

danilofuchs commented 2 months ago

Is there an existing issue for this?

[x] I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
[x] I have reviewed the documentation https://docs.sentry.io/
[x] I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/browser

SDK Version

8.26.0

Framework Version

No response

Link to Sentry event

No response

Reproduction Example/SDK Setup

import {
  getNodeAutoInstrumentations,
  getResourceDetectors,
} from "@opentelemetry/auto-instrumentations-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import * as opentelemetry from "@opentelemetry/sdk-node";
import * as Sentry from "@sentry/node";
import { SentryPropagator, SentrySpanProcessor } from "@sentry/opentelemetry";
import { nodeProfilingIntegration } from "@sentry/profiling-node";

const sdk = new opentelemetry.NodeSDK({
  instrumentations: getNodeAutoInstrumentations(),
  resourceDetectors: getResourceDetectors(),
  spanProcessors: [
    new SentrySpanProcessor(), // Export spans to Sentry
    new opentelemetry.tracing.BatchSpanProcessor(new OTLPTraceExporter()), // Export spans to CloudWatch OLTP sidecar
  ],
  // This conflicts with Sentry's sampling.
  // If we want to enable Sentry tracing, we must
  // use SentrySampler instead of AlwaysOnSampler
  sampler: new opentelemetry.tracing.AlwaysOnSampler(),
  textMapPropagator: new SentryPropagator(),
  contextManager: new Sentry.SentryContextManager(),
});

Sentry.init({
  dsn: "https://dsn",

  // Set tracesSampleRate to 1.0 to capture 100%
  // of transactions for performance monitoring.
  // We recommend adjusting this value in production
  tracesSampleRate: 0,
  profilesSampleRate: 1, // Relative to tracesSampleRate

  environment: process.env.NODE_ENV,
  enabled: process.env.NODE_ENV === "prod",

  integrations: [nodeProfilingIntegration(), Sentry.prismaIntegration()],

  skipOpenTelemetrySetup: true,

  registerEsmLoaderHooks: { exclude: [/openai/] },
});

Sentry.validateOpenTelemetrySetup();

sdk.start();

process.on("SIGTERM", () => {
  sdk
    .shutdown()
    .then(() => console.log("OpenTelemetry SDK terminated"))
    .catch((error) =>
      console.error("Error terminating OpenTelemetry SDK", error),
    );
});

Steps to Reproduce

We are configuring Sentry + OpenTelemetry manually in an ECS container

Our containers were failing to shutdown gracefully on SIGTERM

Expected Result

Document that we should handle shutting down Sentry on SIGTERM, or add a listener in Sentry itself

Something like this solves:

process.on("SIGTERM", () => {
  Sentry.close().then(() => console.log("Sentry stopped"));

  sdk
    .shutdown()
    .then(() => console.log("OpenTelemetry SDK terminated"))
    .catch((error) =>
      console.error("Error terminating OpenTelemetry SDK", error),
    );
});

Actual Result

Application does not exit on SIGTERM, debugging this issue is hard

andreiborza commented 2 months ago

Hello, thanks for writing in. We are on company-wide hackweek and thus on limited support this week. We'll take a look at this next week.

chargome commented 2 months ago

@danilofuchs thanks for pointing that out, I have added a note on our node docs

getsentry / sentry-javascript