search

vercel / next.js

The React Framework
https://nextjs.org
MIT License
124.84k stars 26.65k forks source link

[Next 9] out of memory when building the app #7929

Closed ematipico closed 4 years ago

ematipico commented 5 years ago

Bug report

Describe the bug

Moving an application from Next 8 to Next 9. When I run next build the process goes out of memory and it cannot build the application.

FYI, it's an application with 20 routes more or less.

To Reproduce

That's hard to reproduce as I don't have any idea what went wrong. Next 8 compiles without issue but not Next9.

Here's the stack trace. If you know how to get a more descriptive output, let me know and I will provide:


<--- Last few GCs --->

[28143:0x102634000]   231190 ms: Mark-sweep 1278.7 (1441.0) -> 1263.4 (1438.5) MB, 838.4 / 0.0 ms  (average mu = 0.290, current mu = 0.268) allocation failure scavenge might not succeed
[28143:0x102634000]   231308 ms: Scavenge 1278.1 (1438.5) -> 1266.6 (1440.0) MB, 8.2 / 0.0 ms  (average mu = 0.290, current mu = 0.268) allocation failure 
[28143:0x102634000]   231365 ms: Scavenge 1279.3 (1440.0) -> 1271.6 (1443.0) MB, 21.6 / 0.0 ms  (average mu = 0.290, current mu = 0.268) allocation failure 

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x3547db9dbe3d]
Security context: 0x3a36ebf9e6e1 <JSObject>
    1: DoJoin(aka DoJoin) [0x3a36ebf85e89] [native array.js:~87] [pc=0x3547dcd9a7d0](this=0x3a366f3826f1 <undefined>,l=0x3a368f6eb2b9 <JSArray[10]>,m=10,A=0x3a366f3828c9 <true>,w=0x3a36c3aafc69 <String[1]: />,v=0x3a366f3829a1 <false>)
    2: Join(aka Join) [0x3a36ebf85ed9] [native array.js:~112] [pc=0x3547dd85bb38](this=0x3a366f3826f1 <undefined>,l=0x3a368f6...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10003ae75 node::Abort() [/usr/local/bin/node]
 2: 0x10003b07f node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0x1001a7ae5 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x100572ef2 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/bin/node]
 5: 0x1005759c5 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/bin/node]
 6: 0x10057186f v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
 7: 0x10056fa44 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
 8: 0x10057c2dc v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
 9: 0x10057c35f v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
10: 0x10054e1e4 v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
11: 0x10082784d v8::internal::Runtime_StringBuilderJoin(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
12: 0x3547db9dbe3d 
[1]    28142 abort      npm run build:next

Expected behavior

It should build

Screenshots

If applicable, add screenshots to help explain your problem.

System information

Additional context

Add any other context about the problem here.

timneutkens commented 5 years ago

It's impossible to track this down without a reproduction.

ematipico commented 5 years ago

Yeah I know. Before I was getting a memory heap around the generation of the sourcemaps so I removed it. And I got this error message around the native array file.

Is there some way to get a more verbose stack trace?

ematipico commented 5 years ago

FYI it's now building but only because I added more memory to the node process:

NODE_OPTIONS="--max_old_space_size=4096"

It builds up to 28 routes but when I give 30 routes (we have 31 routes) the memory of the process goes up to 3GB of memory (and even more). I am running the build process on an average laptop that has 8GB of memory, some of it already used. But it managed.

I don't know if I should keep this issue open or not. If it helps, here the stats of the built pages:

image

The process crashes when it tries to create the source maps (most of the times).

timneutkens commented 5 years ago

Really the only way we'll ever know is if a full reproduction is provided. I'll close this for now.

danb235 commented 5 years ago

We too have noticed that after upgrading to Next 9 we are getting out of memory errors whereas with Next 8 this wasn't an issue.

In our case we are building our app in CircleCi as part of our CI flow and encountering the following:

Exited with code 137
Hint: Exit code 137 typically means the process is killed because it was running out of memory
Hint: Check if you can optimize the memory usage in your app
Hint: Max memory usage of this container is 4284268544
 according to /sys/fs/cgroup/memory/memory.max_usage_in_bytes

4GB is memory limit for a CircleCI container, hence the error. It appears building with Next 9 we are hitting that limit.

I don't have a reproducible case to share at the moment.

ZhengYuTay commented 5 years ago

We also faced this problem while deploying my application with NextJs 9.

Out of the 3 projects that we upgraded to NextJs 9, two of them face ram usage problem when deploying to Elastic Beanstalk. End up we have to revert it. Unfortunately I don't have a repo to share to replicate the problem.

stevez86 commented 5 years ago

I'm getting the same out of memory issue, but mine occurs after all lambdas are built. I can finish the build on my laptop, but it crashes when deploying, however I've seen a few out of memory issue periodically while developing locally. The only thing recognizable (to me) is this line:

 readFile [/tmp/b0cc21aa63cece4e/.build-utils/.builder/node_modules/@now/next/dist/index.js:9555]

Here's the log...


Aug 22 2019 01:26:51:346 | λ  (Lambda)       page was emitted as a lambda (i.e. getInitialProps)
-- | --
Aug 22 2019 01:26:51:346 | ⚡  (Static File)  page was prerendered as static HTML
Aug 22 2019 01:26:51:434 | Done in 146.48s.
Aug 22 2019 01:26:51:444 | preparing lambda files...
Aug 22 2019 01:34:22:983 | <--- Last few GCs --->
Aug 22 2019 01:34:22:983 | [430:0x23f5310]   666594 ms: Mark-sweep 4089.3 (4262.1) -> 4089.3 (4261.1) MB, 3898.4 / 0.0 ms  allocation failure GC in old space requested
Aug 22 2019 01:34:22:983 | [430:0x23f5310]   671097 ms: Mark-sweep 4089.3 (4261.1) -> 4089.3 (4225.6) MB, 4502.8 / 0.0 ms  last resort GC in old space requested
Aug 22 2019 01:34:22:983 | [430:0x23f5310]   674990 ms: Mark-sweep 4089.3 (4225.6) -> 4089.3 (4223.1) MB, 3892.4 / 0.0 ms  last resort GC in old space requested
Aug 22 2019 01:34:22:983 | <--- JS stacktrace --->
Aug 22 2019 01:34:22:983 | ==== JS stack trace =========================================
Aug 22 2019 01:34:22:983 | Security context: 0x70e9f6257c1 <JSObject>
Aug 22 2019 01:34:22:983 | 1: toString [buffer.js:611] [bytecode=0x2833662c6061 offset=31](this=0xce2ca70dbc1 <Uint8Array map = 0x6cc06836709>,encoding=0x2fb750b822d1 <undefined>,start=0x2fb750b822d1 <undefined>,end=0x2fb750b822d1 <undefined>)
Aug 22 2019 01:34:22:983 | 2: arguments adaptor frame: 0->3
Aug 22 2019 01:34:22:983 | 3: readFile [/tmp/b0cc21aa63cece4e/.build-utils/.builder/node_modules/@now/next/dist/index.js:9555] [bytecode=0x1814135ff3e1 offset=53](t...
Aug 22 2019 01:34:22:983 | FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
Aug 22 2019 01:34:22:983 | 1:
Aug 22 2019 01:34:22:984 | node::Abort() [/node8/bin/node]
Aug 22 2019 01:34:22:984 | 2:
Aug 22 2019 01:34:22:986 | 0x11e73ec [/node8/bin/node]
Aug 22 2019 01:34:22:986 | 3:
Aug 22 2019 01:34:22:986 | v8::Utils::ReportOOMFailure(char const*, bool) [/node8/bin/node]
Aug 22 2019 01:34:22:986 | 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/node8/bin/node]
Aug 22 2019 01:34:22:986 | 5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/node8/bin/node]
Aug 22 2019 01:34:22:986 | 6:
Aug 22 2019 01:34:22:986 | v8::internal::Factory::NewStringFromUtf8(v8::internal::Vector<char const>, v8::internal::PretenureFlag) [/node8/bin/node]
Aug 22 2019 01:34:22:986 | 7:
Aug 22 2019 01:34:22:987 | v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::NewStringType, int) [/node8/bin/node]
Aug 22 2019 01:34:22:987 | 8:
Aug 22 2019 01:34:22:987 | node::StringBytes::Encode(v8::Isolate*, char const*, unsigned long, node::encoding, v8::Local<v8::Value>*) [/node8/bin/node]
Aug 22 2019 01:34:22:987 | 9:
Aug 22 2019 01:34:22:988 | 0x12070d6 [/node8/bin/node]
Aug 22 2019 01:34:22:988 | 10:
Aug 22 2019 01:34:22:988 | v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) [/node8/bin/node]
Aug 22 2019 01:34:22:988 | 11:
Aug 22 2019 01:34:22:989 | 0xb79dac [/node8/bin/node]
Aug 22 2019 01:34:22:989 | 12:
Aug 22 2019 01:34:22:989 | v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) [/node8/bin/node]
Aug 22 2019 01:34:22:989 | 13: 0x47a70c842fd
Aug 22 2019 01:35:32:549 | done

Here's my webpack config: 

const loadConfig = require('./loadConfig');

const reNodeModules = /node_modules/;
const reScript = /\.(js|jsx|mjs)$/;
const reImage = /\.(bmp|gif|jpg|jpeg|png|svg)$/;
const reGraphql = /\.graphql?$/;
const reStyle = /\.(css|less|styl|scss|sass|sss)$/;

module.exports = (nextConfig = {}) => {
  return {
    ...nextConfig,
    webpack(config, options) {
      const { webpack, isServer, dev } = options;

      const staticAssetName = dev ? '[path][name].[ext]?[hash:8]' : '[name]-[hash:8].[ext]';
      const publicPath = '/_next/static/';
      const outputPath = `${isServer ? '../' : ''}static/`;

      // eslint-disable-next-line no-param-reassign
      config.resolve = {
        ...config.resolve,
        modules: [...config.resolve.modules, './src'],
      };

      config.module.rules.push(
        {
          test: reImage,
          exclude: reNodeModules,
          oneOf: [
            // Inline lightweight into CSS
            {
              issuer: reStyle,
              oneOf: [
                // Inline lightweight SVGs as UTF-8 encoded DataUrl string
                {
                  test: /\.svg$/,
                  loader: 'svg-url-loader',
                  options: {
                    name: staticAssetName,
                    limit: 4096, // 4kb
                    publicPath,
                    outputPath,
                  },
                },

                // Inline lightweight as Base64 encoded DataUrl string
                {
                  loader: 'url-loader',
                  options: {
                    name: staticAssetName,
                    limit: 4096, // 4kb
                    publicPath,
                    outputPath,
                  },
                },
              ],
            },

            // Or return public URL to image resource
            {
              loader: 'file-loader',
              options: {
                name: staticAssetName,
                publicPath,
                outputPath,
              },
            },
          ],
        }, // Rules for GraphQL
        {
          test: reGraphql,
          exclude: reNodeModules,
          use: [
            {
              loader: 'webpack-graphql-loader',
              options: {
                removeUnusedFragments: true,
                output: 'document',
              },
            },
          ],
        },
        {
          exclude: [reNodeModules, reScript, reImage, reGraphql, /\.json$/, /\.txt$/, /\.md$/],
          loader: 'file-loader',
          options: {
            name: staticAssetName,
            publicPath,
            outputPath,
          },
        },
      );

      const appConfig = loadConfig();
      if (isServer && dev) {
        console.info(appConfig);
      }
      config.plugins.push(
        new webpack.DefinePlugin({
          __DEV__: dev,
          __SERVER__: isServer,
          __BROWSER__: !isServer,
          __CONFIG__: JSON.stringify(appConfig),
        }),
      );

      if (typeof nextConfig.webpack === 'function') {
        return nextConfig.webpack(config, options);
      }

      return config;
    },
  };
};
stramel commented 5 years ago

@timneutkens I'm running out of memory using Next.js 9 as well. It could be because of the number of components we are compiling. You mentioned you needed a full repro of it. You can take a look at my branch. https://github.com/stramel/styled-icons/tree/ms/add-material-variants

Here is the failing build: https://travis-ci.org/jacobwgillespie/styled-icons/builds/582987078

Travis has NODE_OPTIONS=--max-old-space-size=4096 set

ematipico commented 5 years ago

@timneutkens could please reopen the issue now that you have a repo?

Clement-Bresson commented 4 years ago

I'm facing the same problem with next@9.0.2.

Timer commented 4 years ago

I'm guessing these application(s) have simply hit their size bounds to start running out of memory. You'll need to run your build with more via NODE_OPTIONS=--max-old-space-size=4096.

Alternatively, you can enable our new chunking feature:

// next.config.js
module.exports = {
  experimental: { granularChunks: true }
}
stramel commented 4 years ago

Thanks @Timer, I will give the granularChunks feature a try.

Also, just to note, I had tried 8192 for the value on my local machine and it still failed. I'm a bit scared to imagine how much memory I would need to actually get it to build.

ematipico commented 4 years ago

I had a look to some old issue and I noticed that sometimes, when there was a new major release, some people faced this memory issue. I wonder, do you do some memory test on some big project?

This new major release is hitting the node memory cap. What's the cause of this big amount memory that wasn't there before?

stramel commented 4 years ago

@Timer I tried using the granularChunks feature and still ran into the same issue. Even locally with 8192 set.

This has the granular chunks config feature set https://github.com/stramel/styled-icons/tree/ms/granular-chunks

Switched to pulling in components dynamically, https://github.com/stramel/styled-icons/tree/ms/dynamic-import-granular-chunks

jrosspaperless commented 4 years ago

This issue happens for one of our repos when we change the build to perform minification, which is disabled by default now.

Update: I determined that in our codebase the issue was using the withSourceMaps plugin:

"@zeit/next-source-maps": "^0.0.4-canary.1"

This makes sense since we have it set to hidden-source-map which is very slow. Though, I don't know why it's exponentially slower than NextJS 8.

mattwills8 commented 4 years ago

We've also noticed massive performance issues after upgrading to v9... I'm using a 2015 macbook pro, it's never lagged before and I've run some pretty heavy programs on it, but now every time i run my next app locally it completely lags out and everything massively slows down.

I usually have to restart the app every 10 minutes or so, today was the first day i tried to persevere and work through it, and after a couple of hours of running in dev move i got that same JavaScript heap out of memory that people have reported from running next build. There must be a memory leak in here somewhere

ematipico commented 4 years ago

I think so too. A colleague of mine has a more recent MacBook and since we upgraded to next 9, the dev server has become slower to recompile. Sometimes it takes up do 20 secs.

xxvvii commented 4 years ago

I'm getting same issue, even tried set NODE_OPTIONS=16384 on my 32GB physic memory MacBook Pro but no effect.

Logs are listed:

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x335bafb5be3d]
Security context: 0x0fe28be1e6e9 <JSObject>
    1: createPrinter [0xfe25fe2c0b9] [/Users/my-name/my-project/node_modules/typescript/lib/typescript.js:~86713] [pc=0x335bb12e86f3](this=0x0fe280c03329 <Object map = 0xfe2df152cc9>,printerOptions=0x0fe2f39e8ae1 <Object map = 0xfe20ac205a9>,handlers=0x0fe28e1026f1 <undefined>)
    2: arguments adaptor frame: 1->2
    3: reportImplicitAny(aka reportImpli...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10003d035 node::Abort() [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 2: 0x10003d23f node::OnFatalError(char const*, char const*) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 3: 0x1001b8e15 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 4: 0x100586d72 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 5: 0x100589845 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 6: 0x1005856ef v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 7: 0x1005838c4 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 8: 0x10059015c v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
 9: 0x1005901df v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
10: 0x10055fb24 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
11: 0x1007e7e04 v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/opt/nvm/versions/node/v10.16.3/bin/node]
12: 0x335bafb5be3d
13: 0x335bb12e86f3
14: 0x335bafb0a5c3
mattwills8 commented 4 years ago

@timneutkens please reopen this issue, or open a new one. This memory leak as well as some weird infinite HMR requests is making our app unusable in development mode and eating up a huge amount of our time.

mtngoatgit commented 4 years ago

Adding another witness. I inherited an app with Next version 4.2.3.

When I did a local serve of the project it failed with the following error: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory The failure usually happened within less than ten minutes of starting the app and could be hastened by triggering more navigation in the app.

I have reverted to version 8.1.0 and cannot reproduce the same failure--even with more earnest navigation (and thus page-builds).

dan-turner commented 4 years ago

Yep, we're seeing this too...

Time to re-open I think @timneutkens

rebotak commented 4 years ago

I'm facing this issue too on next@9.1.3 with react-router custom server since yesterday since I started to import bootstrap.min.css on my development mode.

both when I import directly the bootstrap.min.css with require/import via js or @import via css/scss. my nodejs memory used been increased after few complies while developing.

but when I import the bootstrap from cdn, via in with next/head, the memory seems to fine even after many complies in long developing time.

<Head>
<meta key="viewport" name="viewport" content="initial-scale=1.0, width=device-width" />
<link key="bootstrapcdn" rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" ...  />
...
</Head>

so I think the problem was on the css minification in next-css. because when I downgrade my project to 8.1.0, I got this minification issue. https://github.com/zeit/next-plugins/issues/541. so I tried this workaround (https://github.com/zeit/next-plugins/issues/392#issuecomment-475845330) . and not yet getting the memory issue again.

// next.config.js
const withCSS = require('@zeit/next-css');

function HACK_removeMinimizeOptionFromCssLoaders(config) {
  console.warn(
    'HACK: Removing 'minimize' option from 'css-loader' entries in Webpack config',
  );
  config.module.rules.forEach(rule => {
    if (Array.isArray(rule.use)) {
      rule.use.forEach(u => {
        if (u.loader === 'css-loader' && u.options) {
          delete u.options.minimize;
        }
      });
    }
  });
}

module.exports = withCSS({
  webpack(config) {
    HACK_removeMinimizeOptionFromCssLoaders(config);
    return config;
  },
});
rcmenezes commented 4 years ago

I'm having that same issue after upgrading to Next 9.1.3. I'm using next-css too, maybe that is indeed what's crashing it?

lloan commented 4 years ago

Not entirely sure if this will be the same for everyone, but make sure to check that any resource you are requesting can actually be found. I had an image that my app couldn't find and it kept looking for it until it gave me the heap error. I removed the mention of that image and it hasn't given me the error again.

mpsq commented 4 years ago

I also managed to fix this, @lloan comment helped to debug the issue.

In my case, I was referencing /public/manifest.json as a meta tag in <head/>, however that resource was not present in my project and this was causing the memory leak.

bukinoshita commented 4 years ago

I had this code, which the icon.png doesn't exist

<Head>
  <link rel="icon" href="/static/icon.png" type="image/png" />
</Head>

By fixing the import of the favicon I no longer get the memory leak error

<Head>
  <link rel="icon" href="/icon.png" type="image/png" />
</Head>

As mentioned https://github.com/zeit/now/issues/3307 I believe something's up with static/public folder.

JiDai commented 4 years ago

I had this problem too. As bukinoshita mentioned it, I looked into the static folder. I removed a folder of 674 json files (for testing purposes) out of the public/static/ folder. I did not had the problem since.

focux commented 4 years ago

This issue happens for one of our repos when we change the build to perform minification, which is disabled by default now.

Update: I determined that in our codebase the issue was using the withSourceMaps plugin:

"@zeit/next-source-maps": "^0.0.4-canary.1"

This makes sense since we have it set to hidden-source-map which is very slow. Though, I don't know why it's exponentially slower than NextJS 8.

I started having this issue also after I upgraded to "@zeit/next-source-maps": "^0.0.4-canary.1". Any solution or I'll have to get rid of the source maps?

ematipico commented 4 years ago

@focux I had to disable source maps at all. After that, the usage of memory dropped substantially

webartistse commented 4 years ago

Have same problem here as well. It seems to crash when the file size in build are too big. For example, I imported something from Typescript and the file size in build went up to 2.41 mb. Then my CI with 2GB ram started to crash. After I removed the Typescript import the file size went down to 100kb and it worked again.

Nextjs 9 has been very slow in CI from start. It takes very long to build and I dont have any special needs... just react stuff, material ui, some packages here and there. I have CI:s in same cluster with node/express, some dotnet core, php etc all these has 1GB memory in CI and builds pretty fast. I dont know more than my feeling about the build process has some issue maybe?

neoromantic commented 4 years ago

Same problem here. Mac OS / Github Actions. No actions mentioned here helps, can't find any files that are linked but not existent.

However, project builds (and is in production) in Zeit Now.

Would love to help if knew how.

rigobauer commented 4 years ago

Same problem here (MacOS). What I've found is:

Let me know if I can help. Here you have the error log...

<--- Last few GCs --->

[83442:0x102804000]   123060 ms: Scavenge 1371.3 (1422.4) -> 1370.7 (1422.9) MB, 8.3 / 0.0 ms  (average mu = 0.120, current mu = 0.058) allocation failure 
[83442:0x102804000]   123066 ms: Scavenge 1371.7 (1422.9) -> 1370.9 (1423.9) MB, 4.1 / 0.0 ms  (average mu = 0.120, current mu = 0.058) allocation failure 
[83442:0x102804000]   123071 ms: Scavenge 1371.7 (1423.9) -> 1371.0 (1424.4) MB, 3.9 / 0.0 ms  (average mu = 0.120, current mu = 0.058) allocation failure 

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x3b944125be3d]
Security context: 0x164494d1e6e9 <JSObject>
    1: SourceMapConsumer_allGeneratedPositionsFor [0x16448b97d331] [/Users/alberto.iglesias/Coding/iteisa/projects/ceoe-gis/node_modules/@babel/core/node_modules/source-map/lib/source-map-consumer.js:~178] [pc=0x3b944240968c](this=0x1644193fb679 <BasicSourceMapConsumer map = 0x16448ea8a239>,aArgs=0x16447d082249 <Object map = 0x16448ea98939>)
    2: /* anonym...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10003d035 node::Abort() [/Users/alberto.iglesias/.nvm/versions/node/v10.16.3/bin/node]
 2: 0x10003d23f node::OnFatalError(char const*, char const*) [/Users/alberto.iglesias/.nvm/versions/node/v10.16.3/bin/node]
 3: 0x1001b8e15 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/alberto.iglesias/.nvm/versions/node/v10.16.3/bin/node]
 4: 0x100586d72 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/alberto.iglesias/.nvm/versions/node/v10.16.3/bin/node]
 5: 0x100589845 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/alberto.iglesias/.nvm/versions/node/v10.16.3/bin/node]
 6: 0x1005856ef v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/alberto.iglesias/.nvm/versions/node/v10.16.3/bin/node]
 7: 0x1005838c4 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/alberto.iglesias/.nvm/versions/node/v10.16.3/bin/node]
 8: 0x10059015c v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/alberto.iglesias/.nvm/versions/node/v10.16.3/bin/node]
 9: 0x1005901df v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/alberto.iglesias/.nvm/versions/node/v10.16.3/bin/node]
10: 0x10055fb24 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Users/alberto.iglesias/.nvm/versions/node/v10.16.3/bin/node]
11: 0x1007e7e04 v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/alberto.iglesias/.nvm/versions/node/v10.16.3/bin/node]
12: 0x3b944125be3d 
13: 0x3b944240968c 
Abort trap: 6
ematipico commented 4 years ago

SourceMapConsumer_allGeneratedPositionsFor I think this is the culprit. In dev mode there's a generation of source maps. I think that overtime, this source map plugin retain too much memory.

neoromantic commented 4 years ago

I still experience same problem even if I turn off source map generation

Сергей.

On December 24, 2019 at 10:01 GMT, Emanuele notifications@github.com wrote:

SourceMapConsumer_allGeneratedPositionsFor I think this is the culprit. In dev mode there's a generation of source maps. I think that overtime, these source map plugin retain too much memory.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

timneutkens commented 4 years ago

Keep in mind that posting "I still experience this" won't solve the issue. Provide a full reproduction for people to look into, otherwise your comment is the same as doing a 👍 on the initial issue but pinging everyone in the issue for no reason.

Always make sure your comments are actionable or useful. Eg like ematipico sharing things based on investigating the issue is useful. Saying "I still have this issue" is not useful.

Vista1nik commented 4 years ago

Is there any workaround? My CI/CD Pipeline fails on 1GB of RAM.

dan-turner commented 4 years ago

@timneutkens repro steps are as mentioned in this closed issue:

https://github.com/zeit/next.js/issues/9442#issuecomment-554839437

@Vista1nik workaround that worked for me was to recreate the deprecated static directory.

The related now issue:

https://github.com/zeit/now/issues/3307

tomasfrancisco commented 4 years ago

Just to add to the discussion in case it helps anyone. I was experiencing an out of memory and then I found out what was the issue in my case:

In the following code snippet, if the const Icon happens to be undefined the code just enters in an infinite loop checking if the component is valid. 

const iconMapping = {
  "flash-outline": FlashOutline,
};

export const Icon = ({ name }) => {
  const Icon = iconMapping[name];

  return <Icon />;
}

If I add a check if const Icon is not defined, I get rid of the out of memory issue.

...
  const Icon = iconMapping[name];

  if (!Icon) return null;
  return <Icon />;
}
sandrina-p commented 4 years ago

Same thing happened to me, then I realized was a typo that caused it:

const Button = ({ children }) => {
  // BUG: the button should be lower case (the HTML input)
  //      instead of CamelCase (an undefined component)
  return <Button>{children}</Button>
}
timneutkens commented 4 years ago

Please try upgrading to the latest version of Next.js, we did a bunch of optimizations to memory usage recently.

sandrina-p commented 4 years ago

@timneutkens, I have the latest version (9.3.5). I created the project this morning and faced that error some minutes later.

adam187 commented 4 years ago

Had similar issue on CricleCI while building the app with source maps using next-source-maps

NODE_OPTIONS="--max_old_space_size=4096" helped locally but on CircleCI you also need to increase resource class https://circleci.com/docs/2.0/configuration-reference/#resource_class to large at least to make make it work properly.

laurensnl commented 4 years ago

I started getting this error after updating to next 9.3.6 today:

<--- Last few GCs --->

[66325:0x108008000]    17639 ms: Mark-sweep 1936.2 (2071.4) -> 1923.6 (2073.4) MB, 283.4 / 0.0 ms  (average mu = 0.157, current mu = 0.048) allocation failure scavenge might not succeed
[66325:0x108008000]    17937 ms: Mark-sweep 1938.1 (2073.4) -> 1925.4 (2075.4) MB, 285.8 / 0.0 ms  (average mu = 0.108, current mu = 0.042) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x0fb9ac667d09 <JSObject>
    0: builtin exit frame: stringify(this=0x0fb9ac6598a9 <Object map = 0xfb921b01449>,0x0fb958e804d1 <undefined>,0x0fb958e804d1 <undefined>,0x0fb910180ec1 <Object map = 0xfb9d8491399>,0x0fb9ac6598a9 <Object map = 0xfb921b01449>)

    1: arguments adaptor frame: 1->3
    2: callAsync [0xfb9320a6cf9] [0x0fb958e804d1 <undefined>:~1] [pc=0x1aa09fe84627](this=0x0fb9320a6909 <Hook map = 0xfb9d8495179>...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
...

Fixed it by modifiying paths in tsconfig.json as suggested in #12280

ematipico commented 4 years ago

Probably different cause

szaza commented 4 years ago

The same issue is present here after upgrading from 9.0.5 to 9.3.6.

Creating an optimized production build ..
<--- Last few GCs --->

[1600:0000020DB9FFEBE0]    26245 ms: Mark-sweep 1958.0 (2080.3) -> 1945.0 (2081.5) MB, 266.4 / 0.0 ms  (average mu = 0.179, current mu = 0.077) allocation failure scavenge might not succeed
[1600:0000020DB9FFEBE0]    26530 ms: Mark-sweep 1959.9 (2082.0) -> 1947.2 (2083.8) MB, 265.9 / 0.0 ms  (average mu = 0.127, current mu = 0.068) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 00007FF77B4D4DDD]
Security context: 0x00e7d00c08d1 <JSObject>
    1: /* anonymous */(aka /* anonymous */) [000002BB3E5CAA29] [C:\Users\...\node_modules\enhanced-resolve\lib\UnsafeCachePlugin.js:~27] [pc=00000147695447FC](this=0x00d9404004b1 <undefined>,0x001a9e083681 <Object map = 000003D51551D3A9>,0x001a9e0838d9 <Object map = 000003D515521AE9>,0x001a9e083979 ...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF77A8D363F napi_wrap+128063
 2: 00007FF77A872836 v8::base::CPU::has_sse+35142
 3: 00007FF77A8734F6 v8::base::CPU::has_sse+38406
 4: 00007FF77B089F4E v8::Isolate::ReportExternalAllocationLimitReached+94
 5: 00007FF77B072021 v8::SharedArrayBuffer::Externalize+833
 6: 00007FF77AF3E57C v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1436
 7: 00007FF77AF497D0 v8::internal::Heap::ProtectUnprotectedMemoryChunks+1312
 8: 00007FF77AF462F4 v8::internal::Heap::PageFlagsAreConsistent+3204
 9: 00007FF77AF3BB13 v8::internal::Heap::CollectGarbage+1283
10: 00007FF77AF3A184 v8::internal::Heap::AddRetainedMap+2452
11: 00007FF77AF5C734 v8::internal::Factory::NewInternalizedStringImpl+132
12: 00007FF77AD9C7FD v8::internal::DescriptorArray::Allocate+4941
13: 00007FF77ADAD0C5 v8::internal::StringTable::LookupString+373
14: 00007FF77AAAF356 v8::internal::Factory::InternalizeName+54
15: 00007FF77ACAB0BE v8::internal::Runtime::GetObjectProperty+9662
16: 00007FF77B4D4DDD v8::internal::SetupIsolateDelegate::SetupHeap+546637
17: 00000147695447FC

Any idea?

timneutkens commented 4 years ago

@szaza can you try next@canary, the most likely case is that you have the issue described here: https://github.com/zeit/next.js/issues/12280

szaza commented 4 years ago

Ok, after deleting the node_modules and reinstalling dependencies it seems that it works with the canary version, however I've got now another error: Cannot find module 'next-server/dist/server/next-server'.. Does it seems familiar to you?

szaza commented 4 years ago

Oh I got it, it has been moved to import Server from "next/dist/next-server/server/next-server";. Now it works fine. Thanks for your help!

timneutkens commented 4 years ago

@szaza how did you end up importing that module in your project? It's an internal module? I guess you meant to import 'next' instead?

szaza commented 4 years ago

I'm working on a large project written in TypeScript. There is a passport authentication middleware configured, which redirects the unauthorized users back to the login page. It is implemented the following way:

import Server from "next-server/dist/server/next-server";
...
export const initPassport = (
    server: Express,
    app: Server,
    authStrategies: string[]
) => {
...
return app.render(req, res, AuthRoutes.LOGIN_PAGE);
...
}

After updating next version from 9.0.5 to 9.3.7-canary.0, Server type couldn't be imported anymore from next-server/dist/..., but from from the path mentioned above.