search

gatsbyjs / gatsby

The best React-based framework with performance, scalability and security built in.
https://www.gatsbyjs.com
MIT License
55.27k stars 10.31k forks source link

Building many pages exhausts memory #6349

Closed tfausak closed 6 years ago

tfausak commented 6 years ago

Description

Gatsby is running out of memory when building a site with many (2,500+) pages.

Steps to reproduce

Given the files below, run:

$ env GATSBY_PAGE_COUNT=2600 npm run build

The exact number (2600) required to trigger the failure fluctuates. It's usually around 2556 or so. Using 2500 always succeeds for me, and 2600 always fails.

Expected result

The site should be built.

Actual result

The site fails to build because Node runs out of memory.

$ env GATSBY_PAGE_COUNT=2600 time npm run build 

> @ build /.../example
> rm -f -r .cache public public.tar public.tar.gz && gatsby build && tar -c -f public.tar public && gzip -f -k public.tar

success delete html and css files from previous builds — 0.008 s
success open and validate gatsby-config — 0.004 s
success copy gatsby files — 0.015 s
⠁ Generating 2600 pages ...
success onPreBootstrap — 0.012 s
success source and transform nodes — 0.011 s
success building schema — 0.058 s
success createLayouts — 0.005 s
success createPages — 4.298 s
success createPagesStatefully — 0.205 s
success onPreExtractQueries — 0.000 s
success update schema — 0.112 s
success extract queries from components — 0.010 s
success run graphql queries — 0.703 s
success write out page data — 0.033 s
success write out redirect data — 0.001 s
success onPostBootstrap — 0.000 s

info bootstrap finished - 6.568 s

success Building CSS — 2.919 s
success Building production JavaScript bundles — 26.349 s
⢀ Building static HTML for pages
<--- Last few GCs --->

[23516:0x104002400]    49209 ms: Mark-sweep 951.3 (1453.8) -> 951.3 (1437.8) MB, 122.6 / 0.0 ms  (average mu = 0.006, current mu = 0.000) last resort GC in old space requested
[23516:0x104002400]    49341 ms: Mark-sweep 951.3 (1437.8) -> 951.3 (1437.8) MB, 131.5 / 0.0 ms  (average mu = 0.003, current mu = 0.001) last resort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x33656ef841bd]
Security context: 0x16c4e671e6c9 <JSObject>
    1: DoJoin(aka DoJoin) [0x16c47a34cf41] [native array.js:~87] [pc=0x33656f9fa0a2](this=0x16c4769822e1 <undefined>,l=0x16c4bb354d91 <JSArray[2]>,m=2,A=0x16c476982381 <true>,w=0x16c476982441 <String[0]: >,v=0x16c4769823f1 <false>)
    2: Join(aka Join) [0x16c47a34cf91] [native array.js:~112] [pc=0x33656f9b5534](this=0x16c4769822e1 <undefined>,l=0x16c4bb354d...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: 0x100033aa9 node::Abort() [/usr/local/bin/node]
 2: 0x100035236 node::FatalTryCatch::~FatalTryCatch() [/usr/local/bin/node]
 3: 0x10017a8fb v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x10017a89d v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0x1004532f2 v8::internal::Heap::UpdateSurvivalStatistics(int) [/usr/local/bin/node]
 6: 0x100459823 v8::internal::Heap::AllocateRawWithRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
 7: 0x100439204 v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
 8: 0x100671682 v8::internal::Runtime_StringBuilderConcat(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
 9: 0x33656ef841bd 
sh: line 1: 23516 Abort trap: 6           gatsby build
npm ERR! code ELIFECYCLE
npm ERR! errno 134
npm ERR! @ build: `rm -f -r .cache public public.tar public.tar.gz && gatsby build && tar -c -f public.tar public && gzip -f -k public.tar`
npm ERR! Exit status 134
npm ERR! 
npm ERR! Failed at the @ build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /.../.npm/_logs/2018-07-09T18_22_01_979Z-debug.log
       51.12 real        60.16 user         6.52 sys

Environment

  System:
    OS: macOS High Sierra 10.13.5
    CPU: x64 Intel(R) Core(TM) i5-7267U CPU @ 3.10GHz
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 10.6.0 - /usr/local/bin/node
    npm: 6.1.0 - /usr/local/bin/npm
  Browsers:
    Chrome: 67.0.3396.99
    Firefox: 59.0.2
    Safari: 11.1.1
  npmPackages:
    gatsby: 1.9.273 => 1.9.273 
    gatsby-cli: 1.1.58 => 1.1.58

File contents

package.json:

{
  "dependencies": {
    "gatsby": "1.9.273",
    "gatsby-cli": "1.1.58"
  },
  "scripts": {
    "build": "rm -f -r .cache public public.tar public.tar.gz && gatsby build && tar -c -f public.tar public && gzip -f -k public.tar"
  }
}

gatsby-node.js: 

const path = require('path');
const count = process.env.GATSBY_PAGE_COUNT || 2600;
console.log(`Generating ${count} page${count === 1 ? '' : 's'} ...`);
exports.createPages = (gatsby) => {
  const component = path.resolve('template.js');
  for (let number = 1; number <= count; number += 1) {
    gatsby.boundActionCreators.createPage({
      component,
      path: `/numbers/${number}`,
      context: { number },
    });
  }
};

template.js:

import React from 'react';
export default class NumberPage extends React.Component {
  render() {
    return (
      <p> This is the number page for { this.props.pathContext.number }. </p>
    );
  }
}
KyleAMathews commented 6 years ago

This is a weakness in v1 which we've fixed in v2. See this PR https://github.com/gatsbyjs/gatsby/pull/4912 for details. @pieh found that static-site-generator-webpack-plugin tries to run all HTML builds at once which causes memory usage to spike. I've successfully built 100k pages in this PR https://github.com/gatsbyjs/gatsby/pull/6226 w/o any memory problems.

If you can, we recommend upgrading to v2 https://next.gatsbyjs.org/docs/migrating-from-v1-to-v2/

tfausak commented 6 years ago

Thanks for the quick response! I can confirm that Gatsby 2.0.0-beta.19 is able to build this example without running out of memory. With any luck I will be able to upgrade the real site that was exhibiting this problem to Gatsby v2 🙂

KyleAMathews commented 6 years ago

Great to hear! Please file any issues you find while upgrading to v2! We're working with beta testers to get this ready for GA release.

amykapernick commented 6 years ago

I'm having the same issue and using Gatsby v2. It works locally (although fails every now and then) but is failing on Netlify so I can't actually deploy it

7:23:38 PM: <--- Last few GCs --->
7:23:38 PM: at[1722:0x272ca50]    83661 ms: Mark-sweep 1402.6 (1434.0) -> 1401.9 (1435.5) MB, 2199.1 / 0.0 ms  (+ 41.5 ms in 40 steps since start of marking, biggest step 23.3 ms, walltime since start of marking 2262 ms) (average mu = 0.092, current mu = 0.010) allocat[1722:0x272ca50]    85900 ms: Mark-sweep 1404.4 (1435.5) -> 1403.7 (1437.5) MB, 2140.2 / 0.0 ms  (+ 23.5 ms in 41 steps since start of marking, biggest step 4.0 ms, walltime since start of marking 2239 ms) (average mu = 0.065, current mu = 0.034) allocati
7:23:38 PM: <--- JS stacktrace --->
7:23:38 PM: ==== JS stack trace =========================================
7:23:38 PM:     0: ExitFrame [pc: 0x2e26f77841bd]
7:23:38 PM: Security context: 0x353b4f29e6c9 <JSObject>
7:23:39 PM:     1: /* anonymous */ [0x296001979f51] [/opt/build/repo/node_modules/acorn/dist/acorn.js:~2863] [pc=0x2e26f7f4ed6d](this=0x719e0bec331 <Parser map = 0x34a8fe4e77d1>)
7:23:39 PM:     2: /* anonymous */ [0x296001979659] [/opt/build/repo/node_modules/acorn/dist/acorn.js:~2106] [pc=0x2e26f81e59c0](this=0x719e0bec331 <Parser map = 0x34a8fe4e77d1>,refDestructuringErrors=0x23...
7:23:39 PM: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
7:23:39 PM:  1: 0x8b5f80 node::Abort() [node]
7:23:39 PM:  2: 0x8b5fcc  [node]
7:23:39 PM:  3: 0xab730e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
7:23:39 PM:  4: 0xab7528 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
7:23:39 PM:  5: 0xea5152  [node]
7:23:39 PM:  6: 0xeb10aa v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
7:23:39 PM:  7: 0xeb1a14 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
7:23:39 PM:  8: 0xeb4345 v8::internal::Heap::AllocateRawWithRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [node]
7:23:39 PM:  9: 0xe7c824 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [node]
7:23:39 PM: 10: 0x111e1de v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [node]
7:23:39 PM: 11: 0x2e26f77841bd
7:23:44 PM: Aborted (core dumped)
7:23:44 PM: npm ERR! code ELIFECYCLE
7:23:44 PM: npm ERR! errno 134
7:23:44 PM: npm ERR! amy-goes-to-perth@2.0.0 build: `gatsby build`
7:23:44 PM: npm ERR! Exit status 134

My repo is https://github.com/amykapernick/amygoestoperth/tree/netlify-gatsby

DSchau commented 6 years ago

@amykapernick we're working on this. We think some cache improvements will improve the scalability of Gatsby, and we're hoping to get those in very soon!

amykapernick commented 6 years ago

Thanks @DSchau, my blog isn't even that big either 😬

DSchau commented 6 years ago

@amykapernick yeah, it doesn't really seem like you should've hit the OOM error at the scale of your site. Not really sure what's going on, it's possible some optimizations could be made!

amykapernick commented 6 years ago

@DSchau anything in particular I could try now? Or do I need to wait for changes on your end?

DSchau commented 6 years ago

@amykapernick is it reliably failing in Netlify? I couldn't get it to fail locally, unfortunately!

amykapernick commented 6 years ago

@DSchau every single time. I can get it to fail locally as well the first time I build from scratch (without running develop or anything), but the second time works (whether its because it's partly built). It gives the same error both times.

eg. Just ran a build after deleting the public and .cache folders

success run graphql queries — 39.889 s — 23/23 0.58 queries/second
success write out page data — 0.015 s
success write out redirect data — 0.001 s
success onPostBootstrap — 0.002 s

info bootstrap finished - 77.188 s

⠈ Building production JavaScript and CSS bundles
<--- Last few GCs --->

[17436:000001D7F2E5D860]   104354 ms: Mark-sweep 1392.8 (1424.7) -> 1392.3 (1425.7) MB, 1140.2 / 0.0 ms  (average mu = 0.126, current mu = 0.049) allocation failure scavenge might not succeed
[17436:000001D7F2E5D860]   105320 ms: Mark-sweep 1393.8 (1425.7) -> 1393.3 (1426.7) MB, 954.8 / 0.0 ms  (average mu = 0.075, current mu = 0.012) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 000000A28E6841C1]
Security context: 000003BE0D01E6C9 <JSObject>
    1: /* anonymous */ [0000021C15452751] [C:\Users\amyka\web\amygoestoperth\node_modules\acorn\dist\acorn.js:~2106] [pc=000000A28EFE674B](this=000002886B697D31 <Parser map = 000001E1427B7DB9>,refDestructuringErrors=0000010FCB102201 <null>)
    2: parseExprAtom [000002886B697EA1] [C:\Users\amyka\web\amygoestoperth\node_modules\acorn-dynamic-import\lib\in...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF68E820205
 2: 00007FF68E7FA176
 3: 00007FF68E7FAA50
 4: 00007FF68ED088CE
 5: 00007FF68ED08803
 6: 00007FF68EBE7A04
 7: 00007FF68EBDE4E7
 8: 00007FF68EBDCA6C
 9: 00007FF68EBE56C5
10: 00007FF68EAE9957
11: 00007FF68EC5415A
12: 000000A28E6841C1
npm ERR! code ELIFECYCLE
npm ERR! errno 134
npm ERR! amy-goes-to-perth@2.0.0 build: `gatsby build`
npm ERR! Exit status 134
npm ERR!
npm ERR! Failed at the amy-goes-to-perth@2.0.0 build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\amyka\AppData\Roaming\npm-cache\_logs\2018-10-01T14_11_44_127Z-debug.log

If I run npm run build again (without deleting folders) it works fine

This is my latest failed Netlify build

7:21:05 PM: Build ready to start
7:21:06 PM: build-image version: 42bca793ccd33055023c56c4ca8510463a56d317
7:21:06 PM: buildbot version: 6bab8b64bbd90091082af19fedf16bf73d502e5e
7:21:06 PM: Fetching cached dependencies
7:21:06 PM: Starting to download cache of 85.8MB
7:21:07 PM: Finished downloading cache in 703.088595ms
7:21:07 PM: Starting to extract cache
7:21:10 PM: Finished extracting cache in 2.838516112s
7:21:10 PM: Finished fetching cache in 3.604839172s
7:21:10 PM: Starting to prepare the repo for build
7:21:10 PM: Preparing Git Reference refs/heads/netlify-gatsby
7:21:13 PM: Starting build script
7:21:13 PM: Installing dependencies
7:21:14 PM: Started restoring cached node version
7:21:15 PM: Finished restoring cached node version
7:21:15 PM: Attempting node version 'v10.7.0' from .nvmrc
7:21:16 PM: Downloading and installing node v10.7.0...
7:21:16 PM: Downloading https://nodejs.org/dist/v10.7.0/node-v10.7.0-linux-x64.tar.xz...
7:21:16 PM: 
#                                                                          1.7%
7:21:16 PM: 
##################################
7:21:16 PM:      47.4%
7:21:16 PM: #
7:21:16 PM: ####################################################################### 100.0%
7:21:16 PM: Computing checksum with sha256sum
7:21:16 PM: Checksums matched!
7:21:19 PM: Now using node v10.7.0 (npm v6.1.0)
7:21:19 PM: Attempting ruby version 2.3.6, read from environment
7:21:20 PM: Using ruby version 2.3.6
7:21:20 PM: Using PHP version 5.6
7:21:20 PM: Started restoring cached node modules
7:21:20 PM: Finished restoring cached node modules
7:21:20 PM: Installing NPM modules using NPM version 6.1.0
7:21:30 PM: npm
7:21:30 PM:  WARN deprecated gulp-util@3.0.8: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
7:21:47 PM: > sharp@0.20.8 install /opt/build/repo/node_modules/sharp
7:21:47 PM: > (node install/libvips && node install/dll-copy && prebuild-install) || (node-gyp rebuild && node install/dll-copy)
7:21:47 PM: info
7:21:47 PM:  sharp Detected globally-installed libvips v8.6.2
7:21:47 PM: info sharp Building from source via node-gyp
7:21:49 PM: make: Entering directory `/opt/build/repo/node_modules/sharp/build'
7:21:49 PM:   TOUCH Release/obj.target/libvips-cpp.stamp
7:21:49 PM: Release/obj.target/libvips-cpp.stamp Stats {
7:21:49 PM:   dev: 2064,
7:21:49 PM:   mode: 33188,
7:21:49 PM:   nlink: 1,
7:21:49 PM:   uid: 2500,
7:21:49 PM:   gid: 65534,
7:21:49 PM:   rdev: 0,
7:21:49 PM:   blksize: 4096,
7:21:49 PM:   ino: 1839047,
7:21:49 PM:   size: 0,
7:21:49 PM:   blocks: 0,
7:21:49 PM:   atimeMs: 1538220109000,
7:21:49 PM:   mtimeMs: 1538220109000,
7:21:49 PM:   ctimeMs: 1538220109471.9739,
7:21:49 PM:   birthtimeMs: 1538220109471.9739,
7:21:49 PM:   atime: 2018-09-29T11:21:49.000Z,
7:21:49 PM:   mtime: 2018-09-29T11:21:49.000Z,
7:21:49 PM:   ctime: 2018-09-29T11:21:49.472Z,
7:21:49 PM:   birthtime: 2018-09-29T11:21:49.472Z }
7:21:49 PM:   CXX(target) Release/obj.target/sharp/src/common.o
7:21:51 PM: ./Release/.deps/Release/obj.target/sharp/src/common.o.d.raw Stats {
7:21:51 PM:   dev: 2064,
7:21:51 PM:   mode: 33188,
7:21:51 PM:   nlink: 1,
7:21:51 PM:   uid: 2500,
7:21:51 PM:   gid: 65534,
7:21:51 PM:   rdev: 0,
7:21:51 PM:   blksize: 4096,
7:21:51 PM:   ino: 1839053,
7:21:51 PM:   size: 1431,
7:21:51 PM:   blocks: 8,
7:21:51 PM:   atimeMs: 1538220111000,
7:21:51 PM:   mtimeMs: 1538220111000,
7:21:51 PM:   ctimeMs: 1538220111456.9731,
7:21:51 PM:   birthtimeMs: 1538220111456.9731,
7:21:51 PM:   atime: 2018-09-29T11:21:51.000Z,
7:21:51 PM:   mtime: 2018-09-29T11:21:51.000Z,
7:21:51 PM:   ctime: 2018-09-29T11:21:51.457Z,
7:21:51 PM:   birthtime: 2018-09-29T11:21:51.457Z }
7:21:51 PM:   CXX(target) Release/obj.target/sharp/src/metadata.o
7:21:52 PM: ./Release/.deps/Release/obj.target/sharp/src/metadata.o.d.raw Stats {
7:21:52 PM:   dev: 2064,
7:21:52 PM:   mode: 33188,
7:21:52 PM:   nlink: 1,
7:21:52 PM:   uid: 2500,
7:21:52 PM:   gid: 65534,
7:21:52 PM:   rdev: 0,
7:21:53 PM:   blksize: 4096,
7:21:53 PM:   ino: 1839053,
7:21:53 PM:   size: 1477,
7:21:53 PM:   blocks: 8,
7:21:53 PM:   atimeMs: 1538220112000,
7:21:53 PM:   mtimeMs: 1538220112000,
7:21:53 PM:   ctimeMs: 1538220112891.9724,
7:21:53 PM:   birthtimeMs: 1538220112891.9724,
7:21:53 PM:   atime: 2018-09-29T11:21:52.000Z,
7:21:53 PM:   mtime: 2018-09-29T11:21:52.000Z,
7:21:53 PM:   ctime: 2018-09-29T11:21:52.892Z,
7:21:53 PM:   birthtime: 2018-09-29T11:21:52.892Z }
7:21:53 PM:   CXX(target) Release/obj.target/sharp/src/stats.o
7:21:54 PM: ./Release/.deps/Release/obj.target/sharp/src/stats.o.d.raw Stats {
7:21:54 PM:   dev: 2064,
7:21:54 PM:   mode: 33188,
7:21:54 PM:   nlink: 1,
7:21:54 PM:   uid: 2500,
7:21:54 PM:   gid: 65534,
7:21:54 PM:   rdev: 0,
7:21:54 PM:   blksize: 4096,
7:21:54 PM:   ino: 1839053,
7:21:54 PM:   size: 1468,
7:21:54 PM:   blocks: 8,
7:21:54 PM:   atimeMs: 1538220114000,
7:21:54 PM:   mtimeMs: 1538220114000,
7:21:54 PM:   ctimeMs: 1538220114232.972,
7:21:54 PM:   birthtimeMs: 1538220114232.972,
7:21:54 PM:   atime: 2018-09-29T11:21:54.000Z,
7:21:54 PM:   mtime: 2018-09-29T11:21:54.000Z,
7:21:54 PM:   ctime: 2018-09-29T11:21:54.233Z,
7:21:54 PM:   birthtime: 2018-09-29T11:21:54.233Z }
7:21:54 PM:   CXX(target) Release/obj.target/sharp/src/operations.o
7:21:56 PM: ./Release/.deps/Release/obj.target/sharp/src/operations.o.d.raw Stats {
7:21:56 PM:   dev: 2064,
7:21:56 PM:   mode: 33188,
7:21:56 PM:   nlink: 1,
7:21:56 PM:   uid: 2500,
7:21:56 PM:   gid: 65534,
7:21:56 PM:   rdev: 0,
7:21:56 PM:   blksize: 4096,
7:21:56 PM:   ino: 1839053,
7:21:56 PM:   size: 1465,
7:21:56 PM:   blocks: 8,
7:21:56 PM:   atimeMs: 1538220116000,
7:21:56 PM:   mtimeMs: 1538220116000,
7:21:56 PM:   ctimeMs: 1538220116119.971,
7:21:56 PM:   birthtimeMs: 1538220116119.971,
7:21:56 PM:   atime: 2018-09-29T11:21:56.000Z,
7:21:56 PM:   mtime: 2018-09-29T11:21:56.000Z,
7:21:56 PM:   ctime: 2018-09-29T11:21:56.120Z,
7:21:56 PM:   birthtime: 2018-09-29T11:21:56.120Z }
7:21:56 PM:   CXX(target) Release/obj.target/sharp/src/pipeline.o
7:21:59 PM: ./Release/.deps/Release/obj.target/sharp/src/pipeline.o.d.raw Stats {
7:21:59 PM:   dev: 2064,
7:21:59 PM:   mode: 33188,
7:21:59 PM:   nlink: 1,
7:21:59 PM:   uid: 2500,
7:21:59 PM:   gid: 65534,
7:21:59 PM:   rdev: 0,
7:21:59 PM:   blksize: 4096,
7:21:59 PM:   ino: 1839053,
7:21:59 PM:   size: 1497,
7:21:59 PM:   blocks: 8,
7:21:59 PM:   atimeMs: 1538220119000,
7:21:59 PM:   mtimeMs: 1538220119000,
7:21:59 PM:   ctimeMs: 1538220119475.9695,
7:21:59 PM:   birthtimeMs: 1538220119475.9695,
7:21:59 PM:   atime: 2018-09-29T11:21:59.000Z,
7:21:59 PM:   mtime: 2018-09-29T11:21:59.000Z,
7:21:59 PM:   ctime: 2018-09-29T11:21:59.476Z,
7:21:59 PM:   birthtime: 2018-09-29T11:21:59.476Z }
7:21:59 PM:   CXX(target) Release/obj.target/sharp/src/sharp.o
7:22:00 PM: ./Release/.deps/Release/obj.target/sharp/src/sharp.o.d.raw Stats {
7:22:00 PM:   dev: 2064,
7:22:00 PM:   mode: 33188,
7:22:00 PM:   nlink: 1,
7:22:00 PM:   uid: 2500,
7:22:00 PM:   gid: 65534,
7:22:00 PM:   rdev: 0,
7:22:00 PM:   blksize: 4096,
7:22:00 PM:   ino: 1839053,
7:22:00 PM:   size: 1526,
7:22:00 PM:   blocks: 8,
7:22:00 PM:   atimeMs: 1538220120000,
7:22:00 PM:   mtimeMs: 1538220120000,
7:22:00 PM:   ctimeMs: 1538220120612.969,
7:22:00 PM:   birthtimeMs: 1538220120612.969,
7:22:00 PM:   atime: 2018-09-29T11:22:00.000Z,
7:22:00 PM:   mtime: 2018-09-29T11:22:00.000Z,
7:22:00 PM:   ctime: 2018-09-29T11:22:00.613Z,
7:22:00 PM:   birthtime: 2018-09-29T11:22:00.613Z }
7:22:00 PM:   CXX(target) Release/obj.target/sharp/src/utilities.o
7:22:02 PM: ./Release/.deps/Release/obj.target/sharp/src/utilities.o.d.raw Stats {
7:22:02 PM:   dev: 2064,
7:22:02 PM:   mode: 33188,
7:22:02 PM:   nlink: 1,
7:22:02 PM:   uid: 2500,
7:22:02 PM:   gid: 65534,
7:22:02 PM:   rdev: 0,
7:22:02 PM:   blksize: 4096,
7:22:02 PM:   ino: 1839053,
7:22:02 PM:   size: 1482,
7:22:02 PM:   blocks: 8,
7:22:02 PM:   atimeMs: 1538220122000,
7:22:02 PM:   mtimeMs: 1538220122000,
7:22:02 PM:   ctimeMs: 1538220122045.9685,
7:22:02 PM:   birthtimeMs: 1538220122045.9685,
7:22:02 PM:   atime: 2018-09-29T11:22:02.000Z,
7:22:02 PM:   mtime: 2018-09-29T11:22:02.000Z,
7:22:02 PM:   ctime: 2018-09-29T11:22:02.046Z,
7:22:02 PM:   birthtime: 2018-09-29T11:22:02.046Z }
7:22:02 PM:   SOLINK_MODULE(target) Release/obj.target/sharp.node
7:22:02 PM:   COPY Release/sharp.node
7:22:02 PM: make: Leaving directory `/opt/build/repo/node_modules/sharp/build'
7:22:02 PM: > node-sass@4.9.3 install /opt/build/repo/node_modules/node-sass
7:22:02 PM: > node scripts/install.js
7:22:02 PM: Downloading binary from https://github.com/sass/node-sass/releases/download/v4.9.3/linux-x64-64_binding.node
7:22:04 PM: Download complete
7:22:04 PM: Binary saved to /opt/build/repo/node_modules/node-sass/vendor/linux-x64-64/binding.node
7:22:04 PM: Caching binary to /opt/buildhome/.npm/node-sass/4.9.3/linux-x64-64_binding.node
7:22:04 PM: > pre-commit@1.2.2 install /opt/build/repo/node_modules/pre-commit
7:22:04 PM: > node install.js
7:22:04 PM: > spawn-sync@1.0.15 postinstall /opt/build/repo/node_modules/spawn-sync
7:22:04 PM: > node postinstall
7:22:04 PM: > pngquant-bin@5.0.0 postinstall /opt/build/repo/node_modules/pngquant-bin
7:22:04 PM: > node lib/install.js
7:22:05 PM:   ✔ pngquant pre-build test passed successfully
7:22:05 PM: > cwebp-bin@4.0.0 postinstall /opt/build/repo/node_modules/cwebp-bin
7:22:05 PM: > node lib/install.js
7:22:06 PM:   ✔ cwebp pre-build test passed successfully
7:22:06 PM: > node-sass@4.9.3 postinstall /opt/build/repo/node_modules/node-sass
7:22:06 PM: > node scripts/build.js
7:22:06 PM: Binary found at /opt/build/repo/node_modules/node-sass/vendor/linux-x64-64/binding.node
7:22:06 PM: Testing binary
7:22:06 PM: Binary is fine
7:22:10 PM: npm notice created a lockfile as package-lock.json. You should commit this file.
7:22:10 PM: npm WARN gatsby-source-filesystem@1.5.39 requires a peer of gatsby@^1.9.250 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN netlify-cms-backend-bitbucket@2.0.7 requires a peer of netlify-cms-lib-auth@2.0.0-alpha.0 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN netlify-cms-backend-bitbucket@2.0.7 requires a peer of netlify-cms-lib-util@2.0.0-alpha.0 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN netlify-cms-backend-bitbucket@2.0.7 requires a peer of netlify-cms-ui-default@2.0.0-alpha.0 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN netlify-cms-backend-test@2.0.6 requires a peer of immutable@^3.8.2 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN slate@0.34.7 requires a peer of immutable@>=3.8.1 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN slate-edit-list@0.11.3 requires a peer of immutable@^3.8.2 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN slate-edit-list@0.11.3 requires a peer of slate@^0.32.0 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN slate-edit-table@0.15.2 requires a peer of immutable@^3.8.1 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN slate-edit-table@0.15.2 requires a peer of slate@^0.33.3 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN slate-plain-serializer@0.5.41 requires a peer of immutable@>=3.8.1 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN slate-prop-types@0.4.61 requires a peer of immutable@>=3.8.1 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN slate-react@0.12.9 requires a peer of immutable@>=3.8.1 but none is installed. You must install peer dependencies yourself.
7:22:10 PM: npm WARN amy-goes-to-perth@2.0.0 scripts['tests'] should probably be scripts['test'].
7:22:10 PM: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4 (node_modules/fsevents):
7:22:10 PM: npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
7:22:10 PM: added 650 packages from 375 contributors and audited 50643 packages in 48.827s
7:22:10 PM: found 3 moderate severity vulnerabilities
7:22:10 PM:   run `npm audit fix` to fix them, or `npm audit` for details
7:22:10 PM: NPM modules installed
7:22:10 PM: Started restoring cached go cache
7:22:10 PM: Finished restoring cached go cache
7:22:10 PM: unset GOOS;
7:22:10 PM: unset GOARCH;
7:22:10 PM: export GOROOT='/opt/buildhome/.gimme/versions/go1.10.linux.amd64';
7:22:10 PM: export PATH="/opt/buildhome/.gimme/versions/go1.10.linux.amd64/bin:${PATH}";
7:22:10 PM: go version >&2;
7:22:10 PM: export GIMME_ENV='/opt/buildhome/.gimme/env/go1.10.linux.amd64.env';
7:22:10 PM: go version go1.10 linux/amd64
7:22:10 PM: Installing missing commands
7:22:10 PM: Verify run directory
7:22:10 PM: Executing user command: npm run build
7:22:10 PM: > amy-goes-to-perth@2.0.0 build /opt/build/repo
7:22:10 PM: > gatsby build
7:22:12 PM: success open and validate gatsby-config — 0.010 s
7:22:12 PM: success load plugins — 0.210 s
7:22:13 PM: success onPreInit — 0.775 s
7:22:13 PM: success delete html and css files from previous builds — 0.016 s
7:22:13 PM: success initialize cache — 0.008 s
7:22:13 PM: success copy gatsby files — 0.015 s
7:22:13 PM: success onPreBootstrap — 0.003 s
7:22:13 PM: success source and transform nodes — 0.136 s
7:22:14 PM: success building schema — 0.325 s
7:22:14 PM: success createPages — 0.083 s
7:22:14 PM: success createPagesStatefully — 0.019 s
7:22:14 PM: success onPreExtractQueries — 0.004 s
7:22:14 PM: success update schema — 0.192 s
7:22:14 PM: success extract queries from components — 0.237 s
7:22:15 PM: Process https://twitter.com/DDDPerth/status/1025558951172001793
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/1025583161286131713
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/1025588645544177664
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/1025591034787459072
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/1025601777310228480
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/1025631570286137344
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/1025632220915019776
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/1025637082465628160
7:22:15 PM: Process https://twitter.com/battlepanda_au/status/1025752304325017600
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/975905801519812608
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/975910032297549824
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/975943750080872448
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/975950415110918144
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/975968243130155008
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/975975535456108544
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/937523751003885568
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/937940216257454081
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/937945883450130433
7:22:15 PM: Process https://twitter.com/Amys_Kapers/status/937946043420831745
7:22:15 PM: Process https://twitter.com/juniordev_io/status/936195652966539264
7:22:15 PM: Process https://nodejs.org/en/
7:22:15 PM: No endpoint url for https://nodejs.org/en/
7:22:15 PM: Process https://www.npmjs.com/package/gulp-sass
7:22:15 PM: No endpoint url for https://www.npmjs.com/package/gulp-sass
7:22:15 PM: Process https://www.npmjs.com/package/gulp-replace
7:22:15 PM: No endpoint url for https://www.npmjs.com/package/gulp-replace
7:22:16 PM: Process https://www.npmjs.com/package/gulp-string-replace
7:22:16 PM: No endpoint url for https://www.npmjs.com/package/gulp-string-replace
7:22:16 PM: Process https://www.npmjs.com/package/gulp-sourcemaps
7:22:16 PM: No endpoint url for https://www.npmjs.com/package/gulp-sourcemaps
7:22:34 PM: success run graphql queries — 19.869 s — 23/23 1.16 queries/second
7:22:34 PM: success write out page data — 0.005 s
7:22:34 PM: success write out redirect data — 0.001 s
7:22:34 PM: success onPostBootstrap — 0.001 s
7:22:34 PM: info bootstrap finished - 23.964 s
7:23:38 PM: <--- Last few GCs --->
7:23:38 PM: at[1722:0x272ca50]    83661 ms: Mark-sweep 1402.6 (1434.0) -> 1401.9 (1435.5) MB, 2199.1 / 0.0 ms  (+ 41.5 ms in 40 steps since start of marking, biggest step 23.3 ms, walltime since start of marking 2262 ms) (average mu = 0.092, current mu = 0.010) allocat[1722:0x272ca50]    85900 ms: Mark-sweep 1404.4 (1435.5) -> 1403.7 (1437.5) MB, 2140.2 / 0.0 ms  (+ 23.5 ms in 41 steps since start of marking, biggest step 4.0 ms, walltime since start of marking 2239 ms) (average mu = 0.065, current mu = 0.034) allocati
7:23:38 PM: <--- JS stacktrace --->
7:23:38 PM: ==== JS stack trace =========================================
7:23:38 PM:     0: ExitFrame [pc: 0x2e26f77841bd]
7:23:38 PM: Security context: 0x353b4f29e6c9 <JSObject>
7:23:39 PM:     1: /* anonymous */ [0x296001979f51] [/opt/build/repo/node_modules/acorn/dist/acorn.js:~2863] [pc=0x2e26f7f4ed6d](this=0x719e0bec331 <Parser map = 0x34a8fe4e77d1>)
7:23:39 PM:     2: /* anonymous */ [0x296001979659] [/opt/build/repo/node_modules/acorn/dist/acorn.js:~2106] [pc=0x2e26f81e59c0](this=0x719e0bec331 <Parser map = 0x34a8fe4e77d1>,refDestructuringErrors=0x23...
7:23:39 PM: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
7:23:39 PM:  1: 0x8b5f80 node::Abort() [node]
7:23:39 PM:  2: 0x8b5fcc  [node]
7:23:39 PM:  3: 0xab730e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
7:23:39 PM:  4: 0xab7528 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
7:23:39 PM:  5: 0xea5152  [node]
7:23:39 PM:  6: 0xeb10aa v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
7:23:39 PM:  7: 0xeb1a14 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
7:23:39 PM:  8: 0xeb4345 v8::internal::Heap::AllocateRawWithRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [node]
7:23:39 PM:  9: 0xe7c824 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [node]
7:23:39 PM: 10: 0x111e1de v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [node]
7:23:39 PM: 11: 0x2e26f77841bd
7:23:44 PM: Aborted (core dumped)
7:23:44 PM: npm ERR! code ELIFECYCLE
7:23:44 PM: npm ERR! errno 134
7:23:44 PM: npm ERR! amy-goes-to-perth@2.0.0 build: `gatsby build`
7:23:44 PM: npm ERR! Exit status 134
7:23:44 PM: npm ERR!
7:23:44 PM: npm ERR! Failed at the amy-goes-to-perth@2.0.0 build script.
7:23:44 PM: npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
7:23:44 PM: npm ERR! A complete log of this run can be found in:
7:23:44 PM: npm ERR!     /opt/buildhome/.npm/_logs/2018-09-29T11_23_44_637Z-debug.log
7:23:44 PM: Caching artifacts
7:23:44 PM: Started saving node modules
7:23:44 PM: Finished saving node modules
7:23:44 PM: Started saving pip cache
7:23:44 PM: Finished saving pip cache
7:23:44 PM: Started saving emacs cask dependencies
7:23:44 PM: Finished saving emacs cask dependencies
7:23:44 PM: Started saving maven dependencies
7:23:44 PM: Finished saving maven dependencies
7:23:44 PM: Started saving boot dependencies
7:23:44 PM: Finished saving boot dependencies
7:23:44 PM: Started saving go dependencies
7:23:44 PM: Finished saving go dependencies
7:23:45 PM: Cached node version v10.7.0
7:23:45 PM: Error running command: Build script returned non-zero exit code: 134
7:23:45 PM: Failing build: Failed to build site
7:23:45 PM: failed during stage 'building site': Build script returned non-zero exit code: 134
7:23:45 PM: Finished processing build request in 2m39.065100022s
DSchau commented 6 years ago

@amykapernick it looks like including that (empty) gatsby-remark-oembed was causing some issues with the build, perhaps an infinite loop?

I've successfully deployed your site here and opened a PR against your site with the changes.

amykapernick commented 6 years ago

Thanks but I'm using gatsby-remark-oembed to embed Twitter, Instagram and Codepens in my blog posts from markdown so I need that plugin. I'll have a chat to them and see if they can assist with the issue.

DSchau commented 6 years ago

@amykapernick ah got it. OK so the issue isn't that you're using the plugin, it's that you have an empty plugins/gatsby-remark-oembed folder.

I'd recommend bugging those folks to publish the plugin if they haven't already :)

amykapernick commented 6 years ago

@DSchau I've installed the plugin, I'm using it as a submodule as they're still working on it. I'll have a look and see if I can add it a different way

JoseFMP commented 5 years ago

Did you got to solve this issue? Over here still happening. "gatsby": "^2.3.3"