gastby build failing during "Building production JavaScript and CSS bundles" step - "Callback was already called"

DannyAllegrezza commented 4 years ago

Hi all!

First off, I want to say I am huge fan of Gatsby and promise I spent many, many hours Googling and searching before resorting to opening this issue 😁. I have a personal Gatsby + Netlify blog which I have built off of the gatsby-starter-netlify-cms template. NOTE: I am still using Netlify CMS v3.

Description

The build fails when attempting to execute gatsby build. gatsby develop works perfectly. It appears to be dying on the "Building production JavaScript and CSS bundles" step. Since I use Netlify, my production build logs are public and give me an easy way to try and bisect any changes that could have caused this issue.

10:43:09 AM: success open and validate gatsby-configs - 0.048s
10:43:10 AM: success load plugins - 1.095s
10:43:10 AM: success onPreInit - 0.004s
10:43:10 AM: success delete html and css files from previous builds - 0.013s
10:43:10 AM: success initialize cache - 0.010s
10:43:10 AM: success copy gatsby files - 0.050s
10:43:10 AM: success onPreBootstrap - 0.011s
10:43:10 AM: success createSchemaCustomization - 0.016s
10:43:10 AM: success source and transform nodes - 0.578s
10:43:11 AM: success building schema - 0.648s
10:43:11 AM: success createPages - 0.095s
10:43:11 AM: success createPagesStatefully - 0.148s
10:43:11 AM: success onPreExtractQueries - 0.001s
10:43:12 AM: success update schema - 0.053s
10:43:12 AM: success extract queries from components - 0.573s
10:43:12 AM: success write out requires - 0.004s
10:43:12 AM: success write out redirect data - 0.001s
10:43:12 AM: success onPostBootstrap - 0.001s
10:43:12 AM: ⠀
10:43:12 AM: info bootstrap finished - 5.980 s
10:43:12 AM: ⠀
10:44:16 AM: error UNHANDLED EXCEPTION Callback was already called.
10:44:16 AM: 
10:44:16 AM:   Error: Callback was already called.
10:44:16 AM:   
10:44:16 AM:   - async.js:16 throwError
10:44:16 AM:     [repo]/[neo-async]/async.js:16:11
10:44:16 AM:   
10:44:16 AM:   - async.js:2818 
10:44:16 AM:     [repo]/[neo-async]/async.js:2818:7
10:44:16 AM:   
10:44:16 AM:   - task_queues.js:79 processTicksAndRejections
10:44:16 AM:     internal/process/task_queues.js:79:11
10:44:16 AM:   
10:44:16 AM: 
10:44:16 AM: not finished run queries - 63.444s
10:44:16 AM: not finished Building production JavaScript and CSS bundles - 63.404s
11:11:48 AM: Build exceeded maximum allowed runtime

View the full log here - https://app.netlify.com/sites/cocky-jones-987d7f/deploys/5eeb7d2a167a9b3d4238f66a

I am at a loss at this point. The issue seemingly was correlated with my attempt to add Prism.js to my site in this commit https://github.com/DannyAllegrezza/dannyallegrezza.com/commit/4cbb404004d5a44f2d83e3c5f3ba2f4afbdc5a96 If you look at the Deploy log (https://app.netlify.com/sites/cocky-jones-987d7f/deploys/5e6ee69b93e97200088d5ecb) you'll see that this is the first time this error popped up. I reverted things and the problem persists 😭😭😭

I figure either I uploaded bad data/image, made a bad component (or graphql query) or something similar so I have branched back 10+ commits, cleared the node_modules folder, fresh npm install followed by gatsby build and the problem persists. This seemingly just came out of nowhere and I confirmed that this problem happens also on OSX (13 MBP).

The repo is located here: https://github.com/DannyAllegrezza/dannyallegrezza.com - feel free to clone it and try to perform gatsby build

Last working deploy https://app.netlify.com/sites/cocky-jones-987d7f/deploys/5e6704b70589fa0007142d39 was at this commit https://github.com/DannyAllegrezza/dannyallegrezza.com/commit/1b3ab5958fcfa6e71d64bc8b4400368833a1c966. I have tried to rebase/branch to this commit and run the build command and am still experiencing this error.

I have tried a ton of things at this point:

Attempt to revert code to old working build
Attempt to redeploy a working deploy
Revert packages
Set Node Version to 12 in Netlify
Searched the Interwebs! Found these threads but no resolution
1. https://stackoverflow.com/questions/59575264/building-gatsby-site-on-netlify-err-callback-was-already-called
2. https://github.com/gatsbyjs/gatsby/issues/21977
3. https://community.netlify.com/t/first-time-trying-to-deploy-netlify-cms-gatsby-starter-template-fails/12977/8
4. https://github.com/netlify-templates/gatsby-starter-netlify-cms/issues/76 Operating System: Windows 10 Node: v12.16.1 NPM: 6.13.4
```
λ gatsby --version
Gatsby CLI version: 2.12.46
Gatsby version: 2.23.4
Note: this is the Gatsby version for the site at: E:\Code\Github\dannyallegrezza.com
```

gastby-config.js

module.exports = {
  siteMetadata: {
    title: 'danny allegrezza\'s blog :: software engineering | cars | life',
    description: 'This repo contains an example business website that is built with Gatsby, and Netlify CMS.It follows the JAMstack architecture by using Git as a single source of truth, and Netlify for continuous deployment, and CDN distribution.',
  },
  plugins: [
    'gatsby-plugin-react-helmet',
    'gatsby-plugin-sass',
    {
      // keep as first gatsby-source-filesystem plugin for gatsby image support
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/static/img`,
        name: 'uploads',
      },
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/src/pages`,
        name: 'pages',
      },
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/src/img`,
        name: 'images',
      },
    },
    'gatsby-plugin-sharp',
    'gatsby-transformer-sharp',
    {
      resolve: 'gatsby-transformer-remark',
      options: {
        plugins: [
          `gatsby-remark-autolink-headers`,
          {
            resolve: 'gatsby-remark-relative-images',
            options: {
              name: 'uploads',
            },
          },
          {
            resolve: 'gatsby-remark-images',
            options: {
              // It's important to specify the maxWidth (in pixels) of
              // the content container as this plugin uses this as the
              // base for generating different widths of each image.
              maxWidth: 2048,
            },
          },
          {
            resolve: 'gatsby-remark-copy-linked-files',
            options: {
              destinationDir: 'static',
            }
          }
        ],
      },
    },
    {
      resolve: 'gatsby-plugin-netlify-cms',
      options: {
        modulePath: `${__dirname}/src/cms/cms.js`,
      },
    },
    {
      resolve: 'gatsby-plugin-purgecss', // purges all unused/unreferenced css rules
      options: {
        develop: true,            // Activates purging in npm run develop
        purgeOnly: ['/all.sass'], // applies purging only on the bulma css file
      },
    }, // must be after other CSS plugins
    'gatsby-plugin-netlify', // make sure to keep it last in the array
  ],
}

gatsby-node.js

const _ = require('lodash');
const path = require('path');
const { createFilePath } = require('gatsby-source-filesystem');
const { fmImagesToRelative } = require('gatsby-remark-relative-images');

// createPage() is going to do just that.. generate out pages from any of our discovered
// markdownRemark nodes
// - blog posts
// - car posts
exports.createPages = ({ actions, graphql }) => {
  const { createPage } = actions;

  return graphql(`
    {
      allMarkdownRemark(limit: 1000) {
        edges {
          node {
            id
            fields {
              slug
            }
            frontmatter {
              tags
              templateKey
            }
          }
        }
      }
    }
  `).then(result => {
    if (result.errors) {
      result.errors.forEach(e => console.error(e.toString()));
      return Promise.reject(result.errors);
    }

    const posts = result.data.allMarkdownRemark.edges;

    posts.forEach(edge => {
      const id = edge.node.id;

      createPage({
        path: edge.node.fields.slug,
        tags: edge.node.frontmatter.tags,
        component: path.resolve(
          `src/templates/${String(edge.node.frontmatter.templateKey)}.js`
        ),
        // additional data can be passed via context
        // passing in the "id" property allows for the GraphQL query to find the associated
        // id of the page. See the "cars-page" or "about" page to see how this works
        context: {
          id,
        },
      });
    })

    // Tag pages:
    let tags = [];
    // Iterate through each post, putting all found tags into `tags`
    posts.forEach(edge => {
      if (_.get(edge, `node.frontmatter.tags`)) {
        tags = tags.concat(edge.node.frontmatter.tags);
      }
    })
    // Eliminate duplicate tags
    tags = _.uniq(tags);

    // Make tag pages
    tags.forEach(tag => {
      const tagPath = `/tags/${_.kebabCase(tag)}/`

      createPage({
        path: tagPath,
        component: path.resolve(`src/templates/tags.js`),
        context: {
          tag,
        },
      });
    });
  });
}

// This onCreateNode function will be called by Gatsby whenever a new node is created (or updated)
// We use this API to add the slugs for our markdown pages to MarkdownRemark nodes
// https://www.gatsbyjs.org/tutorial/part-seven/#creating-slugs-for-pages
exports.onCreateNode = ({ node, actions, getNode }) => {
  const { createNodeField } = actions;
  fmImagesToRelative(node) // convert image paths for gatsby images

  if (node.internal.type === `MarkdownRemark`) {
    // The File name of the Markdown file is used as the Slug 
    // createFilePath() handles finding the parent File node along with creating the slug from the Markdown file..
    const slugValue = createFilePath({ node, getNode });

    createNodeField({
      name: `slug`,
      node,
      value: slugValue,
    });
  }
}

package.json

{
  "name": "dannyallegrezza.com",
  "description": "Example Gatsby, and Netlify CMS project",
  "version": "1.1.3",
  "author": "Danny Allegrezza",
  "dependencies": {
    "bulma": "^0.7.5",
    "gatsby": "^2.8.6",
    "gatsby-cli": "^2.7.0",
    "gatsby-image": "^2.1.2",
    "gatsby-plugin-netlify": "^2.0.17",
    "gatsby-plugin-netlify-cms": "^3.0.18",
    "gatsby-plugin-purgecss": "^3.1.1",
    "gatsby-plugin-react-helmet": "^3.0.12",
    "gatsby-plugin-sass": "^2.0.11",
    "gatsby-plugin-sharp": "^2.1.3",
    "gatsby-remark-autolink-headers": "^2.0.16",
    "gatsby-remark-copy-linked-files": "^2.0.13",
    "gatsby-remark-images": "^3.0.14",
    "gatsby-remark-relative-images": "^0.2.3",
    "gatsby-source-filesystem": "^2.0.38",
    "gatsby-transformer-remark": "^2.3.12",
    "gatsby-transformer-sharp": "^2.1.21",
    "gatsby-transformer-yaml": "^2.1.12",
    "gatsby-transformer-yaml-netlify": "^1.0.0-alpha.1",
    "lodash": "^4.17.15",
    "lodash-webpack-plugin": "^0.11.4",
    "netlify-cms": "^2.9.1",
    "node-sass": "^4.12.0",
    "parcel-bundler": "^1.12.3",
    "prop-types": "^15.7.2",
    "react": "^16.8.6",
    "react-dom": "^16.8.6",
    "react-helmet": "^5.2.1",
    "uuid": "^3.2.1"
  },
  "keywords": [
    "gatsby"
  ],
  "license": "MIT",
  "main": "n/a",
  "scripts": {
    "start": "npm run develop",
    "clean": "rimraf .cache public",
    "build": "npm run clean && gatsby build",
    "develop": "npm run clean && gatsby develop",
    "serve": "gatsby serve",
    "format": "prettier --trailing-comma es5 --no-semi --single-quote --write \"{gatsby-*.js,src/**/*.js}\"",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "devDependencies": {
    "prettier": "^1.18.2",
    "rimraf": "^2.6.2"
  }
}

Steps to reproduce

Clone https://github.com/DannyAllegrezza/dannyallegrezza.com

perform npm install
install gatsby cli globally
attempt to run gatsby build

Expected result

Build should run without errors.

Environment

  System:
    OS: Windows 10 10.0.18362
    CPU: (16) x64 AMD Ryzen 7 2700X Eight-Core Processor         
  Binaries:
    Node: 12.16.1 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.12.3 - ~\AppData\Roaming\npm\yarn.CMD
    npm: 6.13.4 - C:\Program Files\nodejs\npm.CMD
  Languages:
    Python: 2.7.17 - /c/Python27/python
  Browsers:
    Edge: 44.18362.449.0
  npmPackages:
    gatsby: ^2.8.6 => 2.23.4 
    gatsby-cli: ^2.7.0 => 2.12.46 
    gatsby-image: ^2.1.2 => 2.2.40 
    gatsby-plugin-netlify: ^2.0.17 => 2.1.32 
    gatsby-plugin-netlify-cms: ^3.0.18 => 3.0.18 
    gatsby-plugin-purgecss: ^3.1.1 => 3.1.1 
    gatsby-plugin-react-helmet: ^3.0.12 => 3.1.22 
    gatsby-plugin-sass: ^2.0.11 => 2.1.28 
    gatsby-plugin-sharp: ^2.1.3 => 2.4.5 
    gatsby-remark-autolink-headers: ^2.0.16 => 2.1.24 
    gatsby-remark-copy-linked-files: ^2.0.13 => 2.1.37 
    gatsby-remark-images: ^3.0.14 => 3.1.44 
    gatsby-remark-relative-images: ^0.2.3 => 0.2.3 
    gatsby-source-filesystem: ^2.0.38 => 2.1.48 
    gatsby-transformer-remark: ^2.3.12 => 2.6.50 
    gatsby-transformer-sharp: ^2.1.21 => 2.3.14 
    gatsby-transformer-yaml: ^2.1.12 => 2.2.24 
    gatsby-transformer-yaml-netlify: ^1.0.0-alpha.1 => 1.0.0-alpha.1 
  npmGlobalPackages:
    gatsby-cli: 2.12.46
    gatsby: 2.1.4

polarathene commented 4 years ago

I can confirm unable to build the project. I installed with yarn and updated packages. I was able to pass the the point that is reported to hang at after the updated packages, however once building pages an error is thrown:

info bootstrap finished - 5.565s

gatsby-plugin-purgecss: Only processing  /clones/dannyallegrezza.com/src/components/all.sass
success Building production JavaScript and CSS bundles - 80.499s
success run page queries - 0.026s - 6/6 228.48/s

gatsby-plugin-purgecss:
 Previous CSS Size: 185.05 KB
 New CSS Size: 18.60 KB (-89.95%)                                                                                                              
 Removed ~166.44 KB of CSS

[                            ]   0.032 s 0/38 0% Building static HTML for pages
/nodejs/14.4.0/.npm/lib/node_modules/gatsby-cli/node_modules/yoga-layout-prebuilt/yoga-layout/build/Release/nbind.js:53
        throw ex;
        ^

TypeError: Unexpected response from worker: undefined
    at ChildProcessWorker.onMessage (/clones/dannyallegrezza.com/node_modules/jest-worker/build/workers/ChildProcessWorker.js:257:15)
    at ChildProcess.emit (events.js:315:20)
    at emit (internal/child_process.js:906:12)
    at processTicksAndRejections (internal/process/task_queues.js:85:21)
{
  childImageSharp: {
    fluid: {
// more image data, followed by cms content nodes

polarathene commented 4 years ago

Someone else chimed in on Discord with this exact error issue(yoga-layout-prebuilt and jest-worker). They had success by reverting their lockfile apparently.

I removed node_modules from my own current project as well as my yarn lock file, pulled in packages again and did gatsby build. Worked fine. Did not work with the project provided here, so it's presumably a bad/incompatible package.

Environment if useful to anyone:

gatsby info --clipboard

  System:
    OS: Linux 5.4 Manjaro Linux
    CPU: (4) x64 Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz
    Shell: 5.0.16 - /bin/bash
  Binaries:
    Node: 14.4.0 - ~/.asdf/installs/nodejs/14.4.0/bin/node
    Yarn: 1.22.4 - ~/.asdf/installs/nodejs/14.4.0/.npm/bin/yarn
    npm: 6.14.5 - ~/.asdf/installs/nodejs/14.4.0/bin/npm
  Languages:
    Python: 3.8.2
  Browsers:
    Firefox: 76.0.1
  npmPackages:
    gatsby: ^2.23.3 => 2.23.7
    gatsby-image: ^2.4.7 => 2.4.7
    gatsby-plugin-sharp: ^2.6.11 => 2.6.12
    gatsby-plugin-styled-components: ^3.3.4 => 3.3.4
    gatsby-source-filesystem: ^2.3.11 => 2.3.12
    gatsby-transformer-json: ^2.4.5 => 2.4.5
    gatsby-transformer-sharp: ^2.5.5 => 2.5.5
  npmGlobalPackages:
    gatsby-cli: 2.12.43

Warnings about outdated dependencies in gatsby when pulling in packages:

info No lockfile found.
[1/4] Resolving packages...
warning gatsby > babel-plugin-add-module-exports > chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
warning gatsby > core-js@2.6.11: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
warning gatsby > @babel/polyfill > core-js@2.6.11: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
warning gatsby > gatsby-cli > core-js@2.6.11: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
warning gatsby > webpack-dev-server > chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
warning gatsby > babel-plugin-add-module-exports > chokidar > fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
warning gatsby > eslint-plugin-graphql > graphql-config > graphql-import@0.7.1: GraphQL Import has been deprecated and merged into GraphQL Tools, so it will no longer get updates. Use GraphQL Tools instead to stay up-to-date! Check out https://www.graphql-tools.com/docs/migration-from-import for migration and https://the-guild.dev/blog/graphql-tools-v6 for new changes.
warning gatsby > gatsby-cli > yurnalist > babel-runtime > core-js@2.6.11: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
warning gatsby > webpack > watchpack > watchpack-chokidar2 > chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
warning gatsby > micromatch > snapdragon > source-map-resolve > resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
warning gatsby > micromatch > snapdragon > source-map-resolve > urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
warning gatsby > gatsby-cli > gatsby-recipes > graphql-tools > @graphql-tools/relay-operation-optimizer > relay-compiler > fbjs > core-js@2.6.11: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
warning gatsby-plugin-sharp > probe-image-size > request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
[2/4] Fetching packages...
info fsevents@2.1.3: The platform "linux" is incompatible with this module.
info "fsevents@2.1.3" is an optional dependency and failed compatibility check. Excluding it from installation.
info fsevents@1.2.13: The platform "linux" is incompatible with this module.
info "fsevents@1.2.13" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning "gatsby > @pmmmwh/react-refresh-webpack-plugin@0.3.3" has incorrect peer dependency "react-refresh@^0.8.2".
warning "gatsby > @typescript-eslint/eslint-plugin > tsutils@3.17.1" has unmet peer dependency "typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta".
warning " > styled-components@5.1.1" has unmet peer dependency "react-is@>= 16.8.0".
warning " > styled-modern-normalize@0.2.0" has incorrect peer dependency "styled-components@^2.0.0 || ^3.2.6 || ^4.1.2".

pieh commented 4 years ago

@polarathene can you try upgrading to gatsby@2.23.7 we fixed at least one case of Unexpected response from worker: undefined there

polarathene commented 4 years ago

@pieh successful build!

I guess that happened while troubleshooting with the 2nd user on Discord as it just started working for them and we had no clue why :sweat_smile:

DannyAllegrezza commented 4 years ago

Okay, going to close this. Huge thank you the Gatsby Discord community and @polarathene for the troubleshooting help! It seems like the issue was likely related to my package-lock.json lockfile getting in some kind of weird state. The fix ended up being:

Fresh reclone of the repo (to ensure the .cache and node_modules folders were gone)
Delete package-lock.json
Updated all packages (used combo of npm outdated and npm-check -u)

It is hard to identify the direct culprit, but ultimately, this seemed to help. I suspect it might have had to do with gatsby-plugin-netlify-cms getting bumped to v4 from v3. For anyone interested, the diff is here:

https://github.com/DannyAllegrezza/dannyallegrezza.com/pull/10/files

gatsbyjs / gatsby