1.9.9 -> 1.9.100, now 'Javascript heap out of memory'

[brandonmp]

I've been putting off upgrading Gatsby b/c I kept getting a 'heap out of memory' error, but I decided today to track it down.

One of my source-filesystem sources creates 656 nodes (of type consultant) from a JSON. the JSON is about 800kb.

This is the output from running yarn develop:

$ yarn develop
yarn run v1.2.1
$ bnr develop
running better-npm-run in /mnt/c/Users/brand/code/admitbrain
Executing script: develop

to be executed: gatsby develop --max-old-space-size=8192
running better-npm-run in /mnt/c/Users/brand/code/admitbrain

success delete html files from previous builds — 0.558 s
success open and validate gatsby-config.js — 0.005 s
success copy gatsby files — 0.178 s
success onPreBootstrap — 0.047 s
⠈ source and transform nodesStarting to fetch data from Contentful
Fetching default locale
⠂ source and transform nodesdefault local is : en-US
⢀ source and transform nodescontentTypes fetched 8
Updated entries  0
Deleted entries  0
Updated assets  0
Deleted assets  0
Fetch Contentful data: 366.713ms
success source and transform nodes — 7.873 s
success building schema — 1.244 s
success createLayouts — 0.050 s
⠠ createPagesStarting: Blog pages.
Finished: Created 11 blog pages.
Starting: Agency pages.
⠠ createPagesFinished: Created 68 Agency pages.
Starting: School pages.
⠐ createPagesFinished: Created 10 School pages.
Starting: Consultant pages.
⠂ createPagesFinished: Created 656 Consultant pages.
success createPages — 16.948 s
success createPagesStatefully — 0.120 s
success onPreExtractQueries — 0.014 s
success update schema — 6.081 s
success extract queries from components — 0.265 s
⠁
<--- Last few GCs --->

[11566:0x37ab5d0]    73868 ms: Mark-sweep 1416.3 (1495.9) -> 1416.3 (1479.9) MB, 915.0 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 915 ms) last resort
[11566:0x37ab5d0]    74780 ms: Mark-sweep 1416.3 (1479.9) -> 1416.3 (1479.9) MB, 912.6 / 0.0 ms  last resort

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x196898628799 <JSObject>
    0: builtin exit frame: captureStackTrace(aka captureStackTrace)(this=0x393b29c02311 <undefined>,0x2f691ed37cd1 <JSFunction CapturedTrace (sfi = 0x25975f15c491)>,0x328575c09329 <CapturedTrace map = 0x38644cc40001>)

    1: _then [/mnt/c/Users/brand/code/admitbrain/node_modules/bluebird/js/release/promise.js:~219] [pc=0x2c92629e6e1f](this=0x3470fb292b09 <Promise map = 0x38644cc400b1>,didFulfil...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x1356bec [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewRawOneByteString(int, v8::internal::PretenureFlag) [node]
 6: v8::internal::Factory::NewStringFromOneByte(v8::internal::Vector<unsigned char const>, v8::internal::PretenureFlag) [node]
 7: v8::internal::Factory::NumberToString(v8::internal::Handle<v8::internal::Object>, bool) [node]
 8: 0x1088f51 [node]
 9: v8::internal::JSStackFrame::ToString() [node]
10: v8::internal::ErrorUtils::FormatStackTrace(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Object>) [node]
11: v8::internal::Builtin_ErrorCaptureStackTrace(int, v8::internal::Object**, v8::internal::Isolate*) [node]
12: 0x2c926270449d
Aborted (core dumped)
error Command failed with exit code 134.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

This happens both with and without the --max-old-space-size=8192 flag.

However, when I remove the pageQuery export from my consultant template, the build completes fine.

I've tried pulling fields from the pageQuery to see if it was just one query that was doing it, but even with a trivial 1-field query, the crash happens.

I can't imagine my 658 nodes is more than Gatsby can handle, but OTOH everything worked fine pre-upgrade (and still does on my Master branch), so just trying to narrow-down cause here.

Here's my versions before the upgrade:

 "gatsby": "^1.9.9",
        "gatsby-image": "^1.0.16",
        "gatsby-link": "^1.6.13",
        "gatsby-plugin-catch-links": "^1.0.6",
        "gatsby-plugin-manifest": "^1.0.7",
        "gatsby-plugin-offline": "^1.0.7",
        "gatsby-plugin-react-helmet": "^1.0.4",
        "gatsby-plugin-resolve-src": "^1.0.0",
        "gatsby-plugin-sharp": "^1.6.3",
        "gatsby-plugin-sitemap": "^1.2.5",
        "gatsby-plugin-typography": "^1.7.7",
        "gatsby-remark-autolink-headers": "^1.4.7",
        "gatsby-remark-external-links": "^0.0.3",
        "gatsby-remark-images": "^1.5.6",
        "gatsby-remark-smartypants": "^1.4.7",
        "gatsby-source-contentful": "^1.3.7",
        "gatsby-source-filesystem": "1.0.1",
        "gatsby-source-google-sheets": "^1.0.5",
        "gatsby-transformer-json": "^1.0.4",
        "gatsby-transformer-remark": "^1.7.4",
        "gatsby-transformer-sharp": "^1.6.2",

And here's my versions post-upgrade:

"gatsby": "^1.9.100",
        "gatsby-image": "^1.0.16",
        "gatsby-link": "^1.6.13",
        "gatsby-plugin-catch-links": "^1.0.6",
        "gatsby-plugin-manifest": "^1.0.7",
        "gatsby-plugin-nprogress": "^1.0.7",
        "gatsby-plugin-offline": "^1.0.7",
        "gatsby-plugin-react-helmet": "^1.0.4",
        "gatsby-plugin-resolve-src": "^1.0.0",
        "gatsby-plugin-sharp": "^1.6.19",
        "gatsby-plugin-sitemap": "^1.2.5",
        "gatsby-plugin-typography": "^1.7.7",
        "gatsby-remark-autolink-headers": "^1.4.7",
        "gatsby-remark-external-links": "^0.0.3",
        "gatsby-remark-images": "^1.5.6",
        "gatsby-remark-smartypants": "^1.4.7",
        "gatsby-source-contentful": "^1.3.19",
        "gatsby-source-filesystem": "^1.5.7",
        "gatsby-source-google-sheets": "^1.0.5",
        "gatsby-transformer-json": "^1.0.11",
        "gatsby-transformer-remark": "^1.7.4",
        "gatsby-transformer-sharp": "^1.6.13",

[brandonmp]

found this comment & realized syntax was incorrect when increasing memory. Tried building w/this command:

node --max-old-space-size=8192 ./node_modules/.bin/gatsby develop

... but still hit 'heap out of memory' error (though took a lot longer).

I'm unable to replicate the error in a start kit w/ my JSON data sources, so I assume it has to be a confluence of factors, but a node process hitting 8GB of RAM on v 1.9.100, but building fine on v 1.9.9, has to be a memory leak somewhere under the hood in Gatsby, yea?

Where in the src can I look to understand what's happening here & how I might alter my configs to work around this? I want in on all the cool stuff added since 1.9.9 :/

[KyleAMathews]

This sounds bad :-( Sorry you're running into this but we appreciate you jumping to help find the fix. We definitely need performance tests in the future.

So best way to narrow down the problem is to use git bisect. Checkout Gatsby source and follow the contributing guide https://www.gatsbyjs.org/docs/how-to-contribute/ to run your code against source.

Then find the git commit id for your current version and start the bisect process to find out when things stopped working.

This isn't a perfect solution since it won't find out if one of our dependencies broke things but as long as the problem is in Gatsby's source, you should be able to track down where things broke fairly quickly.

[szimek]

I'm getting the same error when I try to run gatsby build with about 500 JSON files with filesystem source and json transformer plugins. I'm currently using Gatsby 1.9.112. I'll see if I can figure out what's causing it.

EDIT: It works with Gatsby 1.9.9! With small changes though - I had to force the following versions:

  "resolutions": {
    "graphql": "0.10.5",
    "relay-compiler": "1.3.0"
  },

Otherwise I was getting some errors.

But now the build takes less than 10s:

$ rm -rf .cache/ public/ && gatsby develop
success delete html files from previous builds — 0.007 s
success open and validate gatsby-config.js — 0.004 s
success copy gatsby files — 0.021 s
success source and transform nodes — 1.859 s
success building schema — 0.930 s
success createLayouts — 0.019 s
success createPages — 1.215 s
success createPagesStatefully — 0.029 s
success extract queries from components — 0.174 s
success run graphql queries — 1.997 s
success write out page data — 0.017 s
success update schema — 0.954 s

info bootstrap finished - 9.122 s

I'll try to bump versions and see when it breaks.

[barraq]

@KyleAMathews actually is there a way to see what Gatsby is doing in the background? the --verbose option is not helping as not much is being logged... As @szimek mentioned Gatsby gets stuck in the run graphql queries step, namely Gatsby then becomes unresponsive (need to kill it). Being able to have more information (e.g. --verbose all or something else) would be great. Do you have any idea how to process? I can have a look node debugger like you suggest here https://github.com/gatsbyjs/gatsby/issues/2866 but it might get complex :/

Update: running it with massive heap size unlocked Gatsby node --max-old-space-size=12000 ../../node_modules/gatsby/dist/bin/gatsby.js build 🤔

[szimek]

@KyleAMathews I looked into it and version 1.9.32 works, but 1.9.33 doesn't. It looks like the culprit is commit 7dd5f3b39b2231c91b7d0f410693267cf5575a40. Unfortunately, it added a bunch of recursive functions, so it's not that obvious yet which one(s) are causing it.

[robmccardle]

Running into this issue on my project also. In our context it only kicks in once we have over ~2000 pages on our site & we only require ~1200 for our Jan milestone so we're using as is for now. Will provide details/tests later once we can commit time & resource to fixing but just bumping to let you know others are affected

[benedfit]

I'm also running into this issue when creating ~1700 pages. I've tried my best to optimize the queries on each page as much as possible, but I can't get the run graphql queries step to complete

[szimek]

@robmccardle @benedfit There's already a PR that fixes this issue: https://github.com/gatsbyjs/gatsby/pull/2969. Please +1 it or sth to get it merged sooner :)

I've also created an issue (including very rough implementation) to not inline webpack chunk manifest here: https://github.com/gatsbyjs/gatsby/issues/2992 It can also be something that you might be interested in, if you're trying to use Gatsby with so many pages.

[KyleAMathews]

2969 is out + a handful of other related performance PRs!

Would love to hear from y'all if things work better now. Sorry about the troubles you had in the past.

[szimek]

@KyleAMathews It works fine for us now with ~1K pages, so I think this issue can be closed now.

[DanielRSnell]

I noticed I was having performance issues with page creation using Semantic UI and as soon as I commented it out it started working. I was doing about 1800 pages.

[thundernixon]

I'm running into this issue (or something very similar) on my Gatsby-based blog.

Here's what I get when I run gatsby build:

success delete html and css files from previous builds — 0.096 s
success open and validate gatsby-config.js — 0.007 s
success copy gatsby files — 0.018 s
success onPreBootstrap — 0.364 s
success source and transform nodes — 1.169 s
success building schema — 0.868 s
success createLayouts — 0.012 s
success createPages — 0.773 s
success createPagesStatefully — 0.044 s
success onPreExtractQueries — 0.000 s
success update schema — 1.525 s
success extract queries from components — 0.098 s
success run graphql queries — 8.712 s
success write out page data — 0.005 s
success write out redirect data — 0.001 s
success onPostBootstrap — 0.003 s

info bootstrap finished - 17.226 s

success Building CSS — 9.031 s
⠁ Building production JavaScript bundles
<--- Last few GCs --->

   66181 ms: Mark-sweep 1181.4 (1431.9) -> 1180.8 (1437.9) MB, 103.0 / 0.0 ms [allocation failure] [GC in old space requested].
   66283 ms: Mark-sweep 1180.8 (1437.9) -> 1184.4 (1437.9) MB, 102.3 / 0.0 ms [allocation failure] [GC in old space requested].
   66395 ms: Mark-sweep 1184.4 (1437.9) -> 1199.4 (1405.9) MB, 112.2 / 0.0 ms [last resort gc].
   66505 ms: Mark-sweep 1199.4 (1405.9) -> 1215.7 (1405.9) MB, 109.9 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2fc82293fa99 <JS Object>
    1: stringify [native json.js:~176] [pc=0x14950bb93673] (this=0x2fc822946091 <a JSON with map 0xf4421a13b91>,E=0x200e2d17bfe1 <an Object with map 0x1a0781f9a4d1>,F=0x2fc822904241 <undefined>,S=0x2fc822904241 <undefined>)
    2: arguments adaptor frame: 1->3
    3: /* anonymous */ [/Users/stephennixon/Sites/blog2017-gatsby/node_modules/webpack/lib/SourceMapDevToolPlugin.js:147] [pc=0x14950c9c...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/usr/local/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/usr/local/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/bin/node]
 4: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
 5: v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [/usr/local/bin/node]
 6: v8::internal::BasicJsonStringifier::SerializeString(v8::internal::Handle<v8::internal::String>) [/usr/local/bin/node]
 7: v8::internal::BasicJsonStringifier::Result v8::internal::BasicJsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
 8: v8::internal::BasicJsonStringifier::SerializeJSArraySlow(v8::internal::Handle<v8::internal::JSArray>, unsigned int, unsigned int) [/usr/local/bin/node]
 9: v8::internal::BasicJsonStringifier::Result v8::internal::BasicJsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
10: v8::internal::BasicJsonStringifier::Result v8::internal::BasicJsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [/usr/local/bin/node]
11: v8::internal::Runtime_BasicJSONStringify(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
12: 0x1495078060c7
13: 0x14950bb93673
14: 0x14950781c7d5
[3]    18077 abort      gatsby build

Maybe this is due to my blog being fairly image-heavy? Still, I don't have that many posts, and I can't imagine I have the most image-heavy Gatsby site out there.

Strangely, I ran into this issue yesterday but got past it after updating to Gatsby 1.9.236. However, today I added a couple of images and a bit more text, and the issue is back.

@KyleAMathews I'm not sure I know how to use Git Bisect or to adjust the max-old-space-size, both of which I see mentioned ... are either of these approaches things I should dig deeper into to troubleshoot this?

Thanks for any insight!

[thundernixon]

Strangely, I waited a few days and changed nothing ... and it now works.

In case anyone else comes across this and is in the same bind as me: if I do run into the problem again, I will likely try the suggested fix at https://stackoverflow.com/questions/26094420/fatal-error-call-and-retry-last-allocation-failed-process-out-of-memory/48895989#48895989

[KyleAMathews]

v2 uses a lot less memory as we don't try to render all html files at one 😅

[JoseFMP]

For me this is still happening.

[wontheone1]

@KyleAMathews Could you tell me from what version this was fixed? Our project with "gatsby": "1.9.277" still has the same problem.

[nlicitra]

For anyone that may still be having issues, don't forget to try removing any lock files (package-lock.json or yarn.lock) and doing a fresh install with the most up to date version of gatsby. I spent a good amount of time trying to figure out why I was still getting this error even after updating gatsby and it turned out to be a stale lock file.