search

jlengstorf / netlify-plugin-gatsby-cache

185 stars 21 forks source link

in netlify pipeline: No Gatsby cache found. Building fresh... #2

Closed rickyPanzer closed 4 years ago

rickyPanzer commented 4 years ago

Hi there,

Locally I was able to use to this plugin and it works, but in the netlify pipeline, I'm getting this message:

No Gatsby cache found. Building fresh...

I am also noticing a message above:

6:13:02 PM: Found Netlify Build that didn't match the version latest. Installing now.

It seems to be able to the save the cache properly in the onSaveCache callback, but in a pipeline it's not able to find the cached directory. It seems like it doesn't exist...

console.log("fs.existsSync(netlifyCacheDir)", fs.existsSync(netlifyCacheDir), netlifyCacheDir)
console.log("fs.existsSync(netlifyPublicDir)", fs.existsSync(netlifyPublicDir), netlifyPublicDir)

Logs the following, in the onPreBuild callback:

6:13:26 PM: fs.existsSync(netlifyCacheDir) false /opt/build/cache/gatsby/.cache 6:13:26 PM: fs.existsSync(netlifyPublicDir) false /opt/build/cache/gatsby/public

jlengstorf commented 4 years ago

hmm... something may have changed upstream in the buildbot

I'll take a look at this next week and figure out what’s going on here — thanks!

ehmicky commented 4 years ago

The bug is due to https://github.com/netlify/build/issues/732, which is currently being reviewed.

ehmicky commented 4 years ago

This should be fixed now by https://github.com/netlify/build/pull/739.

rickyPanzer commented 4 years ago

I'm still facing the issue, even when using @jlengstorf 's new PR

3:38:00 PM: Build ready to start
3:38:02 PM: build-image version: 9cade8af58c2cf3a17a1e9433d2e979149488837
3:38:02 PM: build-image tag: v3.3.5
3:38:02 PM: buildbot version: 2fd8f04962d5dbc77b79b9c2f5ab7a077e87376b
3:38:02 PM: Fetching cached dependencies
3:38:02 PM: Starting to download cache of 391.4MB
3:38:06 PM: Finished downloading cache in 4.012766297s
3:38:06 PM: Starting to extract cache
3:38:19 PM: Finished extracting cache in 12.545025097s
3:38:19 PM: Finished fetching cache in 16.865293313s
3:38:19 PM: Starting to prepare the repo for build
3:38:19 PM: Preparing Git Reference pull/93/head
3:38:27 PM: Found Netlify configuration file netlify.toml in site root
3:38:27 PM: Found Netlify configuration file(s). Overriding site configuration
3:38:27 PM: Starting build script
3:38:27 PM: Installing dependencies
3:38:28 PM: Started restoring cached node version
3:38:33 PM: Finished restoring cached node version
3:38:35 PM: v10.19.0 is already installed.
3:38:36 PM: Now using node v10.19.0 (npm v6.13.4)
3:38:37 PM: Attempting ruby version 2.6.2, read from environment
3:38:38 PM: Using ruby version 2.6.2
3:38:38 PM: Using PHP version 5.6
3:38:38 PM: Started restoring cached node modules
3:38:38 PM: Finished restoring cached node modules
3:38:38 PM: Installing NPM modules using NPM version 6.13.4
3:39:08 PM: > node-sass@4.13.0 install /opt/build/repo/node_modules/node-sass
3:39:08 PM: > node scripts/install.js
3:39:08 PM: Downloading binary from https://github.com/sass/node-sass/releases/download/v4.13.0/linux-x64-64_binding.node
3:39:09 PM: Download complete
3:39:09 PM: Binary saved to /opt/build/repo/node_modules/node-sass/vendor/linux-x64-64/binding.node
3:39:09 PM: Caching binary to /opt/buildhome/.npm/node-sass/4.13.0/linux-x64-64_binding.node
3:39:09 PM: > core-js-pure@3.5.0 postinstall /opt/build/repo/node_modules/core-js-pure
3:39:09 PM: > node -e "try{require('./postinstall')}catch(e){}"
3:39:09 PM: Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!
3:39:09 PM: The project needs your help! Please consider supporting of core-js on Open Collective or Patreon: 
3:39:09 PM: > https://opencollective.com/core-js 
3:39:09 PM: > https://www.patreon.com/zloirock 
3:39:09 PM: Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -)
3:39:09 PM: > gatsby-telemetry@1.1.44 postinstall /opt/build/repo/node_modules/gatsby-telemetry
3:39:09 PM: > node src/postinstall.js || true
3:39:09 PM: > gatsby-cli@2.8.18 postinstall /opt/build/repo/node_modules/gatsby/node_modules/gatsby-cli
3:39:09 PM: > node scripts/postinstall.js
3:39:10 PM: > gatsby@2.18.11 postinstall /opt/build/repo/node_modules/gatsby
3:39:10 PM: > node scripts/postinstall.js
3:39:10 PM: > node-sass@4.13.0 postinstall /opt/build/repo/node_modules/node-sass
3:39:10 PM: > node scripts/build.js
3:39:10 PM: Binary found at /opt/build/repo/node_modules/node-sass/vendor/linux-x64-64/binding.node
3:39:10 PM: Testing binary
3:39:11 PM: Binary is fine
3:39:13 PM: npm WARN
3:39:13 PM:  tsutils@3.17.1 requires a peer of typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta but none is installed. You must install peer dependencies yourself.
3:39:13 PM: npm WARN
3:39:13 PM:  optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.10 (node_modules/fsevents):
3:39:13 PM: npm WARN
3:39:13 PM:  notsup
3:39:13 PM:  SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.10: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
3:39:13 PM: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.1.2 (node_modules/chokidar/node_modules/fsevents):
3:39:13 PM: npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
3:39:13 PM: added 49 packages from 17 contributors, removed 271 packages and updated 200 packages in 30.779s
3:39:16 PM: 25 packages are looking for funding
3:39:16 PM:   run `npm fund` for details
3:39:16 PM: NPM modules installed
3:39:16 PM: Started restoring cached go cache
3:39:16 PM: Finished restoring cached go cache
3:39:16 PM: unset GOOS;
3:39:16 PM: unset GOARCH;
3:39:16 PM: export GOROOT='/opt/buildhome/.gimme/versions/go1.12.linux.amd64';
3:39:16 PM: export PATH="/opt/buildhome/.gimme/versions/go1.12.linux.amd64/bin:${PATH}";
3:39:16 PM: go version >&2;
3:39:16 PM: export GIMME_ENV='/opt/buildhome/.gimme/env/go1.12.linux.amd64.env';
3:39:16 PM: go version go1.12 linux/amd64
3:39:16 PM: Installing missing commands
3:39:16 PM: Verify run directory
3:39:19 PM: Found Netlify Build version 0.1.46
3:39:19 PM: Found Netlify Build binary '/opt/buildhome/.nvm/versions/node/v10.19.0/bin/netlify-build'
3:39:20 PM: Found Netlify Build version "0.1.46" that didn't match the required version "latest". Installing Netlify build version "latest" now.
3:39:28 PM: npm
3:39:28 PM: WARN deprecated core-js@2.6.11: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
3:39:35 PM: /opt/buildhome/.nvm/versions/node/v10.19.0/bin/netlify-build -> /opt/buildhome/.nvm/versions/node/v10.19.0/lib/node_modules/@netlify/build/src/core/bin.js
3:39:35 PM: > core-js@2.6.11 postinstall /opt/buildhome/.nvm/versions/node/v10.19.0/lib/node_modules/@netlify/build/node_modules/core-js
3:39:35 PM: > node -e "try{require('./postinstall')}catch(e){}"
3:39:35 PM: > core-js@3.6.4 postinstall /opt/buildhome/.nvm/versions/node/v10.19.0/lib/node_modules/@netlify/build/node_modules/log-process-errors/node_modules/core-js
3:39:35 PM: > node -e "try{require('./postinstall')}catch(e){}"
3:39:35 PM: + @netlify/build@0.1.47
3:39:35 PM: added 1 package, removed 1 package and updated 4 packages in 15.1s
3:39:36 PM: Executing netlify-build command (lifecycle trial enabled): NETLIFY_CONFIG_BUILD_LIFECYCLE_ONBUILD="gatsby build" NETLIFY_CONFIG_BUILD_PUBLISH="public/" netlify-build --site-id "5e97b740-a21e-40e5-9a21-b7f165237ac4"
3:39:37 PM: ❯ Starting Netlify Build v0.1.47
3:39:37 PM:   https://github.com/netlify/build
3:39:37 PM: Flags
3:39:37 PM: { siteId: '5e97b740-a21e-40e5-9a21-b7f165237ac4' }
3:39:37 PM: ❯ Using config file:
3:39:37 PM:   /opt/build/repo/netlify.toml
3:39:39 PM: 
3:39:39 PM: ❯ Loading plugins
3:39:39 PM:   Loading plugin "@netlify/plugin-functions-core" from build core
3:39:39 PM:   Loading plugin from netlify-plugin-gatsby-cache
3:39:39 PM: ❯ Running Netlify Build Lifecycle
3:39:39 PM:   Found 5 commands. Lets do this!
3:39:39 PM: 
3:39:39 PM: ┌──────────────────────────────────────────────────────────────────┐
3:39:39 PM: │ 1. Running onInstall command from @netlify/plugin-functions-core │
3:39:39 PM: └──────────────────────────────────────────────────────────────────┘
3:39:39 PM: 
3:39:39 PM:  ✔  @netlify/plugin-functions-core.onInstall completed in 21ms
3:39:39 PM: 
3:39:39 PM: ┌────────────────────────────────────────────────────────────────┐
3:39:39 PM: │ 2. Running onPreBuild command from netlify-plugin-gatsby-cache │
3:39:39 PM: └────────────────────────────────────────────────────────────────┘
3:39:39 PM: 
3:39:39 PM: { cacheDirs: [ 'public', '.cache' ] }
3:39:39 PM: No Gatsby cache found. Building fresh.
3:39:39 PM:  ✔  netlify-plugin-gatsby-cache.onPreBuild completed in 9ms
3:39:39 PM: 
3:39:39 PM: ┌──────────────────────────────────────────────────────────────────────────┐
3:39:39 PM: │ 3. Running build.lifecycle.onBuild command from netlify.toml config file │
3:39:39 PM: └──────────────────────────────────────────────────────────────────────────┘
3:39:39 PM: 
3:39:39 PM: Running command "gatsby build"
jlengstorf commented 4 years ago

@ehmicky I'm still seeing the problem, too 😕

jlengstorf commented 4 years ago

this should be fixed by #3 — if you install @latest and run a new build with a cleared cache, subsequent builds should work with the cache

if that doesn't work, we found that disconnecting/reconnecting the GitHub repo fixed it on one site that was being weird

rickyPanzer commented 4 years ago

Hey @jlengstorf, I was able to get this working on one of my branches, but then when rebasing another branch on those commits, it was not working on that branch in particular, nor has it worked on the common dev branch, once merged.

Is the caching branch specific? I have yet to understand the lack of consistency of the cache that is downloading from netlify

jlengstorf commented 4 years ago

@rickyPanzer it definitely shouldn't be branch-specific — @ehmicky any ideas on what could be causing this or how to debug it?

jlengstorf commented 4 years ago

@rickyPanzer could you send over a build log? we can reopen this issue if we have a reproduction

thanks!

rickyPanzer commented 4 years ago

sorry wanted to follow up on this. This has been working well for me for the past month, great plugin!