Why do checks keep failing?

[maneesha]

Build checks on PRs keep failing. Here's an example explanation from the Netlify logs (private link). I don't know what this means or how to fix it.

[maneesha]

I think this is a node issue. Netlify says it runs npm install by default.

Maybe @zkamvar can help? (not super urgent but would ideally like to have this fixed before end of year)

[zkamvar]

I also think it's a node issue. Specifically, I think it has to do with the cache that Netlify has (but can not tell how the cache is setup):

For example, in this deploy that worked on November 30, things were built fine and it shows node working to build the dependencies and move everything into folders: https://app.netlify.com/sites/carpentries-org-preview/deploys/63877449d3e970000866bd15#L269-L291

Log showing node installation

```log 7:19:20 AM: No npm workspaces detected 7:19:20 AM: Started restoring cached node modules 7:19:20 AM: Finished restoring cached node modules 7:19:20 AM: Installing NPM modules using NPM version 8.19.2 7:19:21 AM: npm WARN deprecated stable@0.1.8: Modern JS already guarantees Array#sort() is a stable sort, so this library is deprecated. See the compatibility table on MDN: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#browser_compatibility 7:19:22 AM: > carpentries-org@1.0.0 postinstall 7:19:22 AM: > ./postinstall.sh 7:19:22 AM: + rm -rf _assets/stylesheets/bootstrap 7:19:22 AM: + cp -rf node_modules/bootstrap _assets/stylesheets/ 7:19:22 AM: + rm -rf _assets/javascripts/vendor 7:19:22 AM: + mkdir -p _assets/javascripts/vendor/jquery 7:19:22 AM: + mkdir -p _assets/javascripts/vendor/jquery-visible 7:19:22 AM: + mkdir -p _assets/javascripts/vendor/bootstrap 7:19:22 AM: + mkdir -p _assets/javascripts/vendor/popper 7:19:22 AM: + cp -rf node_modules/jquery/dist/jquery.js node_modules/jquery/dist/jquery.min.js node_modules/jquery/dist/jquery.min.map node_modules/jquery/dist/jquery.slim.js node_modules/jquery/dist/jquery.slim.min.js node_modules/jquery/dist/jquery.slim.min.map _assets/javascripts/vendor/jquery/ 7:19:22 AM: + cp -rf node_modules/jquery-visible/benchmark node_modules/jquery-visible/bower.json node_modules/jquery-visible/df-visible.jquery.json node_modules/jquery-visible/examples node_modules/jquery-visible/jquery.visible.js node_modules/jquery-visible/jquery.visible.min.js node_modules/jquery-visible/LICENSE.txt node_modules/jquery-visible/package.json node_modules/jquery-visible/README.markdown _assets/javascripts/vendor/jquery-visible/ 7:19:22 AM: + cp -rf node_modules/bootstrap/dist/js/bootstrap.bundle.js node_modules/bootstrap/dist/js/bootstrap.bundle.js.map node_modules/bootstrap/dist/js/bootstrap.bundle.min.js node_modules/bootstrap/dist/js/bootstrap.bundle.min.js.map node_modules/bootstrap/dist/js/bootstrap.esm.js node_modules/bootstrap/dist/js/bootstrap.esm.js.map node_modules/bootstrap/dist/js/bootstrap.esm.min.js node_modules/bootstrap/dist/js/bootstrap.esm.min.js.map node_modules/bootstrap/dist/js/bootstrap.js node_modules/bootstrap/dist/js/bootstrap.js.map node_modules/bootstrap/dist/js/bootstrap.min.js node_modules/bootstrap/dist/js/bootstrap.min.js.map _assets/javascripts/vendor/bootstrap/ 7:19:22 AM: + cp -rf node_modules/@popperjs/core/dist/cjs node_modules/@popperjs/core/dist/esm node_modules/@popperjs/core/dist/umd _assets/javascripts/vendor/popper/ 7:19:22 AM: added 22 packages, and audited 23 packages in 1s 7:19:22 AM: 10 packages are looking for funding 7:19:22 AM: run `npm fund` for details 7:19:22 AM: found 0 vulnerabilities 7:19:22 AM: NPM modules installed ```

However, recent deploys do not go through all those steps and instead looks for a cache and attempts to restore with no messages:

https://app.netlify.com/sites/carpentries-org-preview/deploys/6387cae43cff26000870e9d4#L139-L141

1:28:15 PM: No npm workspaces detected
1:28:15 PM: Started restoring cached node modules
1:28:15 PM: Finished restoring cached node modules

[zkamvar]

I am rerunning https://app.netlify.com/sites/carpentries-org-preview/deploys/639777eb8a4c33301fac0939 while resetting the cache and it looks to be running, so it is a cache issue. I get a feeling that the JS modules in here are not set up in a typical fashion (probably to facilitate offline development).

[zkamvar]

Also, FWIW, unless something changed, we deploy PR previews through Netlify and https://beta.carpentries.org/ through AWS. The builds from Netlify will go to https://carpentries-org-preview.netlify.app/.

I believe it has something to do with registration of the carpentries domain being tied to CloudFront and being unable to serve a subdomain from another service, but that's an area that I'm still unfamiliar with.

[zkamvar]

I know what's happening:

The cache restore for node modules does not consider the postinstall script in our package.json:

https://github.com/carpentries/beta.carpentries.org/blob/915e51290de2659fe7c7b0450f09f730c37734ab/package.json#L6-L8

When it restore the cache, it's literally just moving the cached files over to node_modules/.

Our solution likely is to run the postinstall script in _build_netlify.sh

[zkamvar]

This explains a bit of the logic behind the caching as well: https://github.com/netlify/build-image/issues/714#issuecomment-998514277

I suspect that the postinstall script that we have right now is a desire path that has not been paved in npm.

[zkamvar]

Note: the path I took to get here:

1. look at the netlify node docs
2. follow link to cache docs
3. Read this text and follow the link

   You can check which directories are cached by searching for $NETLIFY_CACHE_DIR in the run-build-functions.sh file for your site’s selected build image.

4. Follow the link in this text:

   All Netlify build images are stored in a public GitHub repository. The repository README includes a list of available images and instructions for testing locally.

5. Scroll through https://github.com/netlify/build-image/blob/focal/run-build-functions.sh until I found restore_node_modules() and then follow the breadcrumbs from there.