search

gatsbyjs / gatsby

The best React-based framework with performance, scalability and security built in.
https://www.gatsbyjs.com
MIT License
55.28k stars 10.31k forks source link

Error deploying to netlify "not finished Generating image thumbnails - 300.076s "(Build timeout) #23263

Closed nejurgis closed 4 years ago

nejurgis commented 4 years ago

Hey guys, i'm building a site, and it seems that gatsby image is taking too long to process the images, would anyone explain to me what I could do better? Everytime I'm running the build there's more and more image thumbnails, though the ammount of images didnt change.. And at the moment there's less than 100 images, and counting. The build was slow but working until i finally uploaded a lambda function, after that it started always breaking, don't know if it has anything to do with it?

also is it normal for the build for a small website to take 13 min?

Using node v10.18.1

Tried: gatsby-parallel-running Deleting Cache both on netlify and with gatsby clean Switching node versions

and here's the repo: https://github.com/nejurgis/warehouse

Here's my netlify log:

4:36:50 PM: Build ready to start
4:36:52 PM: build-image version: 2dbd444fcdce00cf06325060a8238d5ae3e86774
4:36:52 PM: build-image tag: v3.3.7
4:36:52 PM: buildbot version: 134ec431c6a3ff76e87cc4dc34007fe13842272f
4:36:52 PM: Fetching cached dependencies
4:36:52 PM: Starting to download cache of 1.3GB
4:37:09 PM: Finished downloading cache in 17.47691684s
4:37:09 PM: Starting to extract cache
4:37:45 PM: Finished extracting cache in 35.963482239s
4:37:46 PM: Finished fetching cache in 53.799469871s
4:37:46 PM: Starting to prepare the repo for build
4:37:46 PM: Preparing Git Reference refs/heads/master
4:37:52 PM: Found Netlify configuration file netlify.toml in site root
4:37:52 PM: Found Netlify configuration file(s). Overriding site configuration
4:37:52 PM: Creating functions prep folder
4:37:52 PM: Starting build script
4:37:52 PM: Installing dependencies
4:37:53 PM: Started restoring cached node version
4:37:59 PM: Finished restoring cached node version
4:38:00 PM: v10.20.1 is already installed.
4:38:00 PM: Now using node v10.20.1 (npm v6.14.4)
4:38:01 PM: Attempting ruby version 2.6.2, read from environment
4:38:02 PM: Using ruby version 2.6.2
4:38:03 PM: Using PHP version 5.6
4:38:03 PM: Started restoring cached node modules
4:38:03 PM: Finished restoring cached node modules
4:38:03 PM: Installing NPM modules using NPM version 6.14.4
4:39:13 PM: > deasync@0.1.19 install /opt/build/repo/node_modules/deasync
4:39:13 PM: > node ./build.js
4:39:13 PM: `linux-x64-node-10` exists; testing
4:39:13 PM: Binary is fine; exiting
4:39:13 PM: > sharp@0.25.2 install /opt/build/repo/node_modules/sharp
4:39:13 PM: > (node install/libvips && node install/dll-copy && prebuild-install --runtime=napi) || (node-gyp rebuild && node install/dll-copy)
4:39:14 PM: info
4:39:14 PM: sharp
4:39:14 PM:  Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.9.1/libvips-8.9.1-linux-x64.tar.gz
4:39:16 PM: > node-sass@4.13.1 install /opt/build/repo/node_modules/node-sass
4:39:16 PM: > node scripts/install.js
4:39:17 PM: Downloading binary from https://github.com/sass/node-sass/releases/download/v4.13.1/linux-x64-64_binding.node
4:39:17 PM: Download complete
4:39:17 PM: Binary saved to /opt/build/repo/node_modules/node-sass/vendor/linux-x64-64/binding.node
4:39:18 PM: Caching binary to /opt/buildhome/.npm/node-sass/4.13.1/linux-x64-64_binding.node
4:39:18 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/bmp/node_modules/core-js
4:39:18 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:18 PM: Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!
4:39:18 PM: The project needs your help! Please consider supporting of core-js on Open Collective or Patreon: 
4:39:18 PM: > https://opencollective.com/core-js 
4:39:18 PM: > https://www.patreon.com/zloirock 
4:39:18 PM: Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -)
4:39:18 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/core/node_modules/core-js
4:39:18 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:18 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/custom/node_modules/core-js
4:39:18 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:18 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/gif/node_modules/core-js
4:39:18 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:18 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/jpeg/node_modules/core-js
4:39:18 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:18 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-blit/node_modules/core-js
4:39:18 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:18 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-blur/node_modules/core-js
4:39:18 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:19 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-circle/node_modules/core-js
4:39:19 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:19 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-color/node_modules/core-js
4:39:19 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:19 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-contain/node_modules/core-js
4:39:19 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:19 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-cover/node_modules/core-js
4:39:19 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:19 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-crop/node_modules/core-js
4:39:19 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:19 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-displace/node_modules/core-js
4:39:19 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:19 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-dither/node_modules/core-js
4:39:19 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:20 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-fisheye/node_modules/core-js
4:39:20 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:20 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-flip/node_modules/core-js
4:39:20 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:20 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-gaussian/node_modules/core-js
4:39:20 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:20 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-invert/node_modules/core-js
4:39:20 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:20 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-mask/node_modules/core-js
4:39:20 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:20 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-normalize/node_modules/core-js
4:39:20 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:20 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-print/node_modules/core-js
4:39:20 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:21 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-resize/node_modules/core-js
4:39:21 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:21 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-rotate/node_modules/core-js
4:39:21 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:21 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-scale/node_modules/core-js
4:39:21 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:21 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-shadow/node_modules/core-js
4:39:21 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:21 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugin-threshold/node_modules/core-js
4:39:21 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:21 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/plugins/node_modules/core-js
4:39:21 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:21 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/png/node_modules/core-js
4:39:21 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:21 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/tiff/node_modules/core-js
4:39:21 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:22 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/types/node_modules/core-js
4:39:22 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:22 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/@jimp/utils/node_modules/core-js
4:39:22 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:22 PM: > core-js@2.6.11 postinstall /opt/build/repo/node_modules/core-js
4:39:22 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:22 PM: > core-js-pure@3.6.5 postinstall /opt/build/repo/node_modules/core-js-pure
4:39:22 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:22 PM: > core-js@3.6.5 postinstall /opt/build/repo/node_modules/jimp/node_modules/core-js
4:39:22 PM: > node -e "try{require('./postinstall')}catch(e){}"
4:39:22 PM: > gatsby-telemetry@1.2.5 postinstall /opt/build/repo/node_modules/gatsby-telemetry
4:39:22 PM: > node src/postinstall.js || true
4:39:22 PM: > gatsby-cli@2.11.10 postinstall /opt/build/repo/node_modules/gatsby/node_modules/gatsby-cli
4:39:22 PM: > node scripts/postinstall.js
4:39:23 PM: > gatsby@2.20.24 postinstall /opt/build/repo/node_modules/gatsby
4:39:23 PM: > node scripts/postinstall.js
4:39:23 PM: > node-sass@4.13.1 postinstall /opt/build/repo/node_modules/node-sass
4:39:23 PM: > node scripts/build.js
4:39:23 PM: Binary found at /opt/build/repo/node_modules/node-sass/vendor/linux-x64-64/binding.node
4:39:23 PM: Testing binary
4:39:24 PM: Binary is fine
4:39:24 PM: > styled-components@4.4.1 postinstall /opt/build/repo/node_modules/styled-components
4:39:24 PM: > node ./scripts/postinstall.js || exit 0
4:39:24 PM: Use styled-components at work? Consider supporting our development efforts at https://opencollective.com/styled-components
4:39:28 PM: npm WARN react-codemirror2@6.0.1 requires a peer of codemirror@5.x but none is installed. You must install peer dependencies yourself.
4:39:28 PM: npm WARN
4:39:28 PM:  react-redux@4.4.10 requires a peer of redux@^2.0.0 || ^3.0.0 but none is installed. You must install peer dependencies yourself.
4:39:28 PM: npm WARN
4:39:28 PM:  optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.12 (node_modules/fsevents):
4:39:28 PM: npm WARN notsup
4:39:28 PM:  SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.12: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
4:39:28 PM: npm WARN
4:39:28 PM:  optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.1.2 (node_modules/chokidar/node_modules/fsevents):
4:39:28 PM: npm WARN
4:39:28 PM:  notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
4:39:28 PM: added 769 packages from 376 contributors, removed 757 packages, updated 693 packages and audited 61184 packages in 84.421s
4:39:32 PM: 154 packages are looking for funding
4:39:32 PM:   run `npm fund` for details
4:39:32 PM: found 6 low severity vulnerabilities
4:39:32 PM:   run `npm audit fix` to fix them, or `npm audit` for details
4:39:32 PM: NPM modules installed
4:39:32 PM: Started restoring cached go cache
4:39:32 PM: Finished restoring cached go cache
4:39:32 PM: unset GOOS;
4:39:32 PM: unset GOARCH;
4:39:32 PM: export GOROOT='/opt/buildhome/.gimme/versions/go1.12.linux.amd64';
4:39:32 PM: export PATH="/opt/buildhome/.gimme/versions/go1.12.linux.amd64/bin:${PATH}";
4:39:32 PM: go version >&2;
4:39:32 PM: export GIMME_ENV='/opt/buildhome/.gimme/env/go1.12.linux.amd64.env';
4:39:32 PM: go version go1.12 linux/amd64
4:39:32 PM: Installing missing commands
4:39:32 PM: Verify run directory
4:39:33 PM: Executing user command: gatsby-parallel-runner
4:39:38 PM: success open and validate gatsby-configs - 0.057s
4:39:40 PM: success load plugins - 1.769s
4:39:40 PM: plugin-netlify-cache: Restoring 0 cached files for .cache directory with 2 already existing files.
4:39:40 PM: plugin-netlify-cache: Netlify cache restored
4:39:40 PM: success onPreInit - 0.022s
4:39:40 PM: success delete html and css files from previous builds - 0.017s
4:39:40 PM: success initialize cache - 0.011s
4:39:40 PM: success copy gatsby files - 0.052s
4:39:40 PM: success onPreBootstrap - 0.005s
4:39:40 PM: Initializing gatsby-remark-component
4:39:40 PM: success createSchemaCustomization - 0.031s
4:39:43 PM: success source and transform nodes - 2.657s
4:39:44 PM: success building schema - 0.471s
4:39:44 PM: success createPages - 0.102s
4:39:44 PM: success createPagesStatefully - 0.122s
4:39:44 PM: success onPreExtractQueries - 0.001s
4:39:44 PM: success update schema - 0.031s
4:39:44 PM: success extract queries from components - 0.510s
4:39:44 PM: warning The GraphQL query in the non-page component "/opt/build/repo/src/templates/podcast-page.js" will not be run.
4:39:44 PM: Exported queries are only executed for Page components. It's possible you're
4:39:44 PM: trying to create pages in your gatsby-node.js and that's failing for some
4:39:44 PM: reason.
4:39:44 PM: If the failing component(s) is a regular component and not intended to be a page
4:39:44 PM: component, you generally want to use a <StaticQuery> (https://gatsbyjs.org/docs/static-query)
4:39:44 PM: instead of exporting a page query.
4:39:44 PM: If you're more experienced with GraphQL, you can also export GraphQL
4:39:44 PM: fragments from components and compose the fragments in the Page component
4:39:44 PM: query and pass data down into the child component — https://graphql.org/learn/queries/#fragments
4:39:44 PM: success write out requires - 0.006s
4:39:44 PM: success write out redirect data - 0.001s
4:39:44 PM: success onPostBootstrap - 0.001s
4:39:44 PM: ⠀
4:39:44 PM: info bootstrap finished - 8.275 s
4:39:44 PM: ⠀
4:42:32 PM: success Building production JavaScript and CSS bundles - 64.794s
4:42:34 PM: success Rewriting compilation hashes - 0.001s
4:42:41 PM: warning You can't use childImageSharp together with undefined.undefined — use publicURL instead. The childImageSharp portion of the query in this file will return null:
4:42:41 PM: /opt/build/repo/src/templates/about-page.js
4:42:41 PM: success run queries - 176.330s - 64/64 0.36/s
4:44:45 PM: Error during processing Job timed out 8da66e2d-4485-4da8-a950-a39797fe5586
4:44:45 PM: Processing failed 8da66e2d-4485-4da8-a950-a39797fe5586  error: Job timed out 8da66e2d-4485-4da8-a950-a39797fe5586
4:44:45 PM: error Job timed out 8da66e2d-4485-4da8-a950-a39797fe5586
4:44:45 PM: 
4:44:45 PM:   WorkerError: Job timed out 8da66e2d-4485-4da8-a950-a39797fe5586
4:44:45 PM:   
4:44:45 PM:   - jobs-manager.js:314 exports.enqueueJob
4:44:45 PM:     [repo]/[gatsby]/dist/utils/jobs-manager.js:314:23
4:44:45 PM:   
4:44:45 PM:   - next_tick.js:68 process._tickCallback
4:44:45 PM:     internal/process/next_tick.js:68:7
4:44:45 PM:   
4:44:45 PM: 
4:44:45 PM: not finished Generating image thumbnails - 300.076s
4:44:45 PM: Error during processing Job timed out 7633439b-793a-4889-bc6f-b6addfb31e29
4:44:45 PM: Processing failed 7633439b-793a-4889-bc6f-b6addfb31e29  error: Job timed out 7633439b-793a-4889-bc6f-b6addfb31e29
4:44:45 PM: Error during processing Job timed out 562cd646-2271-4b63-bc0b-84140e86ab6d
4:44:45 PM: Processing failed 562cd646-2271-4b63-bc0b-84140e86ab6d  error: Job timed out 562cd646-2271-4b63-bc0b-84140e86ab6d
4:44:45 PM: Function Dir: /opt/build/repo/functions
4:44:45 PM: TempDir: /tmp/zisi-5e9b1082566f400006384f5e
4:44:46 PM: Prepping functions with zip-it-and-ship-it 0.3.1
4:44:47 PM: [ { path: '/tmp/zisi-5e9b1082566f400006384f5e/subscribe.zip',
4:44:47 PM:     runtime: 'js' } ]
4:44:47 PM: Prepping functions complete
4:44:47 PM: Caching artifacts
4:44:47 PM: Started saving node modules
4:44:47 PM: Finished saving node modules
4:44:47 PM: Started saving pip cache
4:44:47 PM: Finished saving pip cache
4:44:47 PM: Started saving emacs cask dependencies
4:44:47 PM: Finished saving emacs cask dependencies
4:44:47 PM: Started saving maven dependencies
4:44:47 PM: Finished saving maven dependencies
4:44:47 PM: Started saving boot dependencies
4:44:47 PM: Finished saving boot dependencies
4:44:47 PM: Started saving go dependencies
4:44:47 PM: Finished saving go dependencies
4:44:47 PM: Error running command: Build script returned non-zero exit code: 1
4:44:47 PM: Failing build: Failed to build site
4:44:47 PM: failed during stage 'building site': Build script returned non-zero exit code: 1
4:44:47 PM: Finished processing build request in 7m55.487577557s

p.s it works when building locally

LekoArts commented 4 years ago

would anyone explain to me what I could do better?

You could preoptimize your images (manually or automatically) so that the resulting images are smaller/quicker to process: https://www.gatsbyjs.org/docs/preoptimizing-images/ - this is especially relevant if you e.g. have 50 megapixel images.

You could also change the srcSetBreakpoints option to create less images or for only certain breakpoints: https://www.gatsbyjs.org/docs/gatsby-image/#images-that-stretch-across-a-fluid-container

Everytime I'm running the build there's more and more image thumbnails, though the ammount of images didnt change

This sounds not correct, can you give a reproduction of that?

The build was slow but working until i finally uploaded a lambda function, after that it started always breaking, don't know if it has anything to do with it?

If this issue is solely related to gatsby-parallel-runner we sadly can't help you with that. You'll need to contact Netlify for that.

github-actions[bot] commented 4 years ago

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open! As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

github-actions[bot] commented 4 years ago

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it. Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! 💪💜