JavaScript heap out of memory

foodiecompany commented 5 months ago

Is there an existing issue for this?

[X] I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
[X] I have reviewed the documentation https://docs.sentry.io/
[X] I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/nextjs

SDK Version

7.117.0

Framework Version

@sentry/nextjs

Link to Sentry event

No response

SDK Setup

Sentry.init({
  enabled,

  environment: IS_PRODUCTION ? "production" : "development",

  dsn: "",

  // Adjust this value in production, or use tracesSampler for greater control
  tracesSampleRate: 1,

  // Setting this option to true will print useful information to the console while you're setting up Sentry.
  debug: false,

  replaysOnErrorSampleRate: 1.0,

  // This sets the sample rate to be 10%. You may want this to be 100% while
  // in development and sample at a lower rate in production
  replaysSessionSampleRate: 0.1,

  // You can remove this option if you're not planning to use the Sentry Session Replay feature:
  integrations: [
    Sentry.replayIntegration({
      // Additional Replay configuration goes in here, for example:
      maskAllText: true,
      blockAllMedia: true
    })
  ]
});

Steps to Reproduce

npm run build

Expected Result

#26 3.523 staff:build: warn  - In order to be able to deminify errors, @sentry/nextjs creates sourcemaps and uploads them to the Sentry server. Depending on your deployment setup, this means your original code may be visible in browser devtools in production. To prevent this, set hideSourceMaps to true in the sentry options in your next.config.js. To disable this warning without changing sourcemap behavior, set hideSourceMaps to false. (In @sentry/nextjs version 8.0.0 and beyond, this option will default to true.) See https://webpack.js.org/configuration/devtool/ and https://docs.sentry.io/platforms/javascript/guides/nextjs/manual-setup/#use-hidden-source-map for more information.
#26 3.523 staff:build: 
#26 172.2 staff:build: 
#26 172.2 staff:build: <--- Last few GCs --->
#26 172.2 staff:build: 
#26 172.2 staff:build: [53:0x7f1f45da40c0]   166470 ms: Mark-sweep 2016.6 (2086.5) -> 1969.1 (2049.4) MB, 1677.0 / 0.0 ms  (average mu = 0.174, current mu = 0.141) allocation failure; scavenge might not succeed
#26 172.2 staff:build: [53:0x7f1f45da40c0]   168145 ms: Mark-sweep 2008.0 (2072.5) -> 1975.8 (2061.5) MB, 1553.9 / 0.0 ms  (average mu = 0.128, current mu = 0.073) allocation failure; scavenge might not succeed
#26 172.2 staff:build: 
#26 172.2 staff:build: 
#26 172.2 staff:build: <--- JS stacktrace --->
#26 172.2 staff:build: 
#26 172.2 staff:build: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Actual Result

The build is successful when Sentry is disabled.

AbhiPrasad commented 5 months ago

Hi @foodiecompany could you share your next.config.js file?

The issue here is probably sourcemaps generation (which is handled by nextjs, sentry just is responsible for uploading). What version of next.js/webpack are you using?

foodiecompany commented 5 months ago

Sure

const { withSentryConfig } = require("@sentry/nextjs");
const path = require("path");

const env = process.env;
const SENTRY_PROJECT_NAME = env.SENTRY_PROJECT_NAME;
const IS_PRODUCTION = env.NODE_ENV === "production";
const STRAPI_MEDIA_HOST = IS_PRODUCTION ? "res.cloudinary.com" : "127.0.0.1";

let config = {
  reactStrictMode: true,
  output: "standalone",
  transpilePackages: [
    "ui",
    "@ant-design",
    "antd",
    "rc-pagination",
    "rc-picker",
    "rc-util"
  ],
  images: {
    remotePatterns: [
      {
        protocol: IS_PRODUCTION ? "https" : "http",
        hostname: STRAPI_MEDIA_HOST
      }
    ]
  },
  experimental: {
    outputFileTracingRoot: path.join(__dirname, "../../"),
    optimizePackageImports: ["lodash", "@ant-design", "antd"],

    // The instrumentation hook is required for Sentry to work on the serverside
    instrumentationHook: true
  }
};

// Injected content via Sentry wizard below
if (SENTRY_PROJECT_NAME) {
  config = withSentryConfig(config, {
    // For all available options, see:
    // https://github.com/getsentry/sentry-webpack-plugin#options

    org: "fastybill-vr",
    project: SENTRY_PROJECT_NAME,

    // Only print logs for uploading source maps in CI
    silent: true,

    // Upload a larger set of source maps for prettier stack traces (increases build time)
    widenClientFileUpload: true,

    // Transpiles SDK to be compatible with IE11 (increases bundle size)
    transpileClientSDK: true,

    // Route browser requests to Sentry through a Next.js rewrite to circumvent ad-blockers.
    // This can increase your server load as well as your hosting bill.
    // Note: Check that the configured route will not match with your Next.js middleware, otherwise reporting of client-
    // side errors will fail.
    tunnelRoute: "/monitoring",

    // Hides source maps from generated client bundles
    hideSourceMaps: true,

    // Automatically tree-shake Sentry logger statements to reduce bundle size
    disableLogger: true,

    // Enables automatic instrumentation of Vercel Cron Monitors.
    // See the following for more information:
    // https://docs.sentry.io/product/crons/
    // https://vercel.com/docs/cron-jobs
    automaticVercelMonitors: true
  });
}

module.exports = config;

Dockerfile

FROM node:18-alpine AS base
RUN apk update && apk add --no-cache build-base gcc autoconf automake zlib-dev libpng-dev vips-dev git > /dev/null 2>&1

FROM base AS builder
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk update
RUN apk add --no-cache libc6-compat
# Set working directory
WORKDIR /app
RUN yarn global add turbo
COPY . .
RUN turbo prune staff --docker

# Add lockfile and package.json's of isolated subworkspace
FROM base AS installer
RUN apk update
RUN apk add --no-cache libc6-compat
WORKDIR /app

# First install the dependencies (as they change less often)
COPY .gitignore .gitignore
COPY --from=builder /app/out/json/ .
COPY --from=builder /app/out/package-lock.json ./package-lock.json
RUN npm install -g node-gyp
RUN npm i sharp
RUN npm install

# Build the project
COPY --from=builder /app/out/full/ .
COPY turbo.json turbo.json

# The SENTRY env are send when we deployed the docker image
ARG SENTRY_PROJECT_NAME
ENV SENTRY_PROJECT_NAME=${SENTRY_PROJECT_NAME}

ARG SENTRY_AUTH_TOKEN
ENV SENTRY_AUTH_TOKEN=${SENTRY_AUTH_TOKEN}

ARG SENTRY_DSN
ENV SENTRY_DSN=${SENTRY_DSN}

ENV NODE_ENV=production
ENV NEXT_TELEMETRY_DISABLED=1

RUN npm run build:staff

FROM base AS runner
WORKDIR /app

# Don't run production as root
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
USER nextjs

COPY --from=builder /app/apps/staff/next.config.js .
COPY --from=installer /app/apps/staff/package.json .

# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=installer --chown=nextjs:nodejs /app/apps/staff/.next/standalone ./
COPY --from=installer --chown=nextjs:nodejs /app/apps/staff/.next/static ./apps/staff/.next/static
COPY --from=installer --chown=nextjs:nodejs /app/apps/staff/public ./apps/staff/public

# Expose ports (for orchestrators and dynamic reverse proxies)
EXPOSE 3001
ENV PORT=3001
ENV HOSTNAME="0.0.0.0"

# Run the nextjs app
CMD ["node", "apps/staff/server.js"]

AbhiPrasad commented 5 months ago

Can you try this workaround? https://github.com/getsentry/sentry-javascript/issues/10468#issuecomment-2004710692

foodiecompany commented 5 months ago

Can you try this workaround? #10468 (comment)

works thx

{
  // ...
  disableServerWebpackPlugin: true,
  disableClientWebpackPlugin: true
}

leighkendell commented 5 months ago

I'm seeing similar issues in version 8.11.0, specifically when building within a docker container. This version doesn't have the same disableServerWebpackPlugin and disableClientWebpackPlugin options but I've tried with:

unstable_sentryWebpackPluginOptions: {
  disable: true
}

which doesn't seem to help. Builds without the withSentryConfig wrapper work fine.

lforst commented 5 months ago

This is very likely because webpack is consuming a lot of memory generating sourcemaps. You can set the following in version 8.x:

module.exports = withSentryConfig(nextConfig, {
  sourcemaps: {
    disable: true
  }
})

pascuflow commented 3 months ago

On @sentry/nextjs 8.26.0 , Next.js 14.2.5 , and adding sourcemaps: { disable: true } still getting heap memory error.


    widenClientFileUpload: true,
    sourcemaps: { disable: true },
    hideSourceMaps: true,
    disableLogger: true,
    automaticVercelMonitors: true,

chargome commented 3 months ago

@pascuflow exporting your next.js config without Sentry does not cause any errors? Can you share your next.config.js?

pascuflow commented 3 months ago

@chargome Correct, I had to remove Sentry so my vercel prod build would work. Here's the next.config:

const path = require("path");
const withPWA = require("next-pwa")({
  disable: process.env.NODE_ENV === "development",
  dest: "public",
  register: true,
  sw: "/sw.js",
});
const removeImports = require("next-remove-imports")();
const { withSentryConfig } = require("@sentry/nextjs");

const securityHeaders = [
  { key: "X-DNS-Prefetch-Control", value: "on" },
  { key: "Strict-Transport-Security", value: "max-age=63072000; includeSubDomains; preload" },
  { key: "X-XSS-Protection", value: "1; mode=block" },
  { key: "SameSite", value: "none" },
];

module.exports = withSentryConfig(
  removeImports(
    withPWA({
      swcMinify: true,
      webpack: true,
      productionBrowserSourceMaps: true,
      experimental: {
        serverMinification: false,
      },
      images: {
        remotePatterns: [
          {
            protocol: "https",
            hostname: "**",
            port: "",
            pathname: "/**",
          },
        ],
      },
      async headers() {
        return [
          { source: "/:path*", headers: securityHeaders },
        ];
      },
      sassOptions: {
        includePaths: [path.join(__dirname, "styles")],
      },
      env: {
        ENV: process.env.NODE_ENV,
      },
    })
  ),
  {
    org: "test",
    project: "test-nextjs",
    silent: !process.env.CI,
    widenClientFileUpload: true,
    reactComponentAnnotation: { enabled: false },
    sourcemaps: {
      disable: true,
    },
    hideSourceMaps: true,
    disableLogger: true,
    automaticVercelMonitors: true,
  }
);

And the build logs:

Run vercel build --prod --token=***
Vercel CLI 36.0.0
Installing dependencies...
npm warn deprecated inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
npm warn deprecated npmlog@5.0.1: This package is no longer supported.
npm warn deprecated rimraf@3.0.2: Rimraf versions prior to v4 are no longer supported.
npm warn deprecated are-we-there-yet@2.0.0: This package is no longer supported.
npm warn deprecated gauge@3.0.2: This package is no longer supported.
npm warn deprecated sourcemap-codec@1.4.8: Please use @jridgewell/sourcemap-codec instead.
npm warn deprecated rollup-plugin-terser@7.0.2: This package has been deprecated and is no longer maintained. Please use @rollup/plugin-terser.
npm warn deprecated workbox-google-analytics@6.6.0: It is not compatible with newer versions of GA starting with v4, as long as you are using GAv3 it should be ok, but the package is no longer being maintained.
npm warn deprecated workbox-cacheable-response@6.6.0: workbox-background-sync@6.6.0
npm warn deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm warn deprecated glob@7.1.7: Glob versions prior to v9 are no longer supported.
npm warn deprecated domexception@4.0.0: Use your platform's native DOMException instead.
npm warn deprecated abab@2.0.6: Use your platform's native atob() and btoa() methods instead.
npm warn deprecated @humanwhocodes/object-schema@2.0.3: Use @eslint/object-schema instead.
npm warn deprecated @humanwhocodes/config-array@0.11.14: Use @eslint/config-array instead.
npm warn deprecated rimraf@2.7.1: Rimraf versions prior to v4 are no longer supported.

added 2018 packages in 47s

497 packages are looking for funding
  run `npm fund` for details
Detected Next.js version: 14.2.5
Detected `package-lock.json` generated by npm 7+
Running "npm run build"

> project@2.0.2 build
> next build

Attention: Next.js now collects completely anonymous telemetry regarding usage.
This information is used to shape Next.js' roadmap and prioritize features.
You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
https://nextjs.org/telemetry

  ▲ Next.js 14.2.5
  - Experiments (use with caution):
    · serverMinification
    · instrumentationHook

   Linting and checking validity of types ...
   Creating an optimized production build ...
> [PWA] Compile server
> [PWA] Compile server
> [PWA] Compile client (static)
> [PWA] Auto register service worker
> [PWA] Service worker: /public/sw.js
> [PWA]   url: /sw.js
> [PWA]   scope: /
[@sentry/nextjs - Node.js] Info: Sending telemetry data on issues and performance to Sentry. To disable telemetry, set `options.telemetry` to `false`.
[@sentry/nextjs - Edge] Info: Sending telemetry data on issues and performance to Sentry. To disable telemetry, set `options.telemetry` to `false`.
[@sentry/nextjs - Client] Info: Sending telemetry data on issues and performance to Sentry. To disable telemetry, set `options.telemetry` to `false`.
[BABEL] Note: The code generator has deoptimized the styling of a large file as it exceeds the max of 500KB.
[BABEL] Note: The code generator has deoptimized the styling of another large file as it exceeds the max of 500KB.

<--- Last few GCs --->

[1725:0x69568b0]   159920 ms: Mark-sweep 2014.4 (2087.8) -> 2006.3 (2088.1) MB, 1561.9 / 0.0 ms  (average mu = 0.171, current mu = 0.086) allocation failure; scavenge might not succeed
[1725:0x69568b0]   161694 ms: Mark-sweep 2014.7 (2088.3) -> 2006.5 (2088.6) MB, 1626.9 / 0.0 ms  (average mu = 0.128, current mu = 0.083) allocation failure; scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb9c1f0 node::Abort() [node]
 2: 0xaa27ee  [node]
 3: 0xd73950 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xd73cf7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xf51075  [node]
 6: 0xf51f78 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]
 7: 0xf62473  [node]
 8: 0xf632e8 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xfd2574 v8::internal::ScavengeJob::Task::RunInternal() [node]
10: 0xe4304b non-virtual thunk to v8::internal::CancelableTask::Run() [node]
11: 0xc07144  [node]
12: 0xc0a5ae node::PerIsolatePlatformData::FlushForegroundTasksInternal() [node]
13: 0x16754b6  [node]
14: 0x1687a24  [node]
15: 0x1675e1e uv_run [node]
16: 0xad9a4a node::SpinEventLoop(node::Environment*) [node]
17: 0xbe1844 node::NodeMainInstance::Run() [node]
18: 0xb54dc8 node::LoadSnapshotDataAndRun(node::SnapshotData const**, node::InitializationResult const*) [node]
19: 0xb58a2f node::Start(int, char**) [node]
20: 0x7f9aa4c29d90  [/lib/x86_64-linux-gnu/libc.so.6]
21: 0x7f9aa4c29e40 __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6]
22: 0xad789e _start [node]
Aborted (core dumped)
Error: Command "npm run build" exited with 134
Error: Process completed with exit code 1.

Lms24 commented 3 months ago

We'll continue to look into this next week as this week is Hackweek at Sentry (see #13421)

chargome commented 3 months ago

@pascuflow could try updating your build command and see if that fixes it for now?

pascuflow commented 3 months ago

Don't think I can control NODE_OPTIONS on Vercel.

lforst commented 3 months ago

Pretty sure you can provide the env var in the interface you configure the build script with :thinking:

pascuflow commented 3 months ago

Pretty sure you can provide the env var in the interface you configure the build script with 🤔

You're right, directly in the build script worked.

"build": "NODE_OPTIONS=\"--max-old-space-size=4096\" next build"

chargome commented 3 months ago

@pascuflow did it actually resolve your issue?

pascuflow commented 3 months ago

@pascuflow did it actually resolve your issue?

Yes, build was successful with repo on Github and Vercel deployment. Logged an error in production and saw it on the Issues tab along with source mapping.

getsantry[bot] commented 1 week ago

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

getsentry / sentry-javascript