search

zooniverse / front-end-monorepo

A rebuild of the front-end for zooniverse.org
https://www.zooniverse.org
Apache License 2.0
105 stars 29 forks source link

yarn bootstrap errors on MacOS while building the canvas dependency #6391

Open eatyourgreens opened 3 days ago

eatyourgreens commented 3 days ago

Describe the bug

yarn bootstrap fails on MacOS Sonoma when it tries to run post-install builds for the canvas package. My local build environment is:

Here's the build output with the error.

[5/5] 🔨  Building fresh packages...
[10/12] â „ @newrelic/native-metrics
[-/12] â „ waiting...
[12/12] â „ core-js
[4/12] â „ canvas
error /Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas: Command failed.
Exit code: 1
Command: node-pre-gyp install --fallback-to-build --update-binary
Arguments: 
Directory: /Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas
Output:
node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using node-pre-gyp@1.0.11
node-pre-gyp info using node@20.13.1 | darwin | arm64
node-pre-gyp http GET https://github.com/Automattic/node-canvas/releases/download/v2.11.2/canvas-v2.11.2-node-v115-darwin-unknown-arm64.tar.gz
node-pre-gyp ERR! install response status 404 Not Found on https://github.com/Automattic/node-canvas/releases/download/v2.11.2/canvas-v2.11.2-node-v115-darwin-unknown-arm64.tar.gz 
node-pre-gyp WARN Pre-built binaries not installable for canvas@2.11.2 and node@20.13.1 (node-v115 ABI, unknown) (falling back to source compile with node-gyp) 
node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/Automattic/node-canvas/releases/download/v2.11.2/canvas-v2.11.2-node-v115-darwin-unknown-arm64.tar.gz 
gyp info it worked if it ends with ok
gyp info using node-gyp@10.0.1
gyp info using node@20.13.1 | darwin | arm64
gyp info ok 
gyp info it worked if it ends with ok
gyp info using node-gyp@10.0.1
gyp info using node@20.13.1 | darwin | arm64
gyp info find Python using Python version 3.11.5 found at "/Users/jimodonnell/anaconda3/bin/python3"
gyp info spawn /Users/jimodonnell/anaconda3/bin/python3
gyp info spawn args [
gyp info spawn args '/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/Users/jimodonnell/Library/Caches/node-gyp/20.13.1/include/node/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/Users/jimodonnell/Library/Caches/node-gyp/20.13.1',
gyp info spawn args '-Dnode_gyp_dir=/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp',
gyp info spawn args '-Dnode_lib_file=/Users/jimodonnell/Library/Caches/node-gyp/20.13.1/<(target_arch)/node.lib',
gyp info spawn args '-Dmodule_root_dir=/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.'
gyp info spawn args ]

/bin/sh: pkg-config: command not found
gyp: Call to 'pkg-config pixman-1 --libs' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack at ChildProcess.<anonymous> (/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp/lib/configure.js:271:18)
gyp ERR! stack at ChildProcess.emit (node:events:519:28)
gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Darwin 23.6.0
gyp ERR! command "/Users/jimodonnell/.nvm/versions/node/v20.13.1/bin/node" "/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--update-binary" "--module=/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas/build/Release/canvas.node" "--module_name=canvas" "--module_path=/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas/build/Release" "--napi_version=9" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v115"
gyp ERR! cwd /Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas
gyp ERR! node -v v20.13.1
gyp ERR! node-gyp -v v10.0.1
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute '/Users/jimodonnell/.nvm/versions/node/v20.13.1/bin/node /Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --update-binary --module=/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas/build/Release/canvas.node --module_name=canvas --module_path=/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas/build/Release --napi_version=9 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v115' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/@mapbox/node-pre-gyp/lib/util/compile.js:89:23)
node-pre-gyp ERR! stack     at ChildProcess.emit (node:events:519:28)
node-pre-gyp ERR! stack     at maybeClose (node:internal/child_process:1105:16)
node-pre-gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:305:5)
node-pre-gyp ERR! System Darwin 23.6.0
node-pre-gyp ERR! command "/Users/jimodonnell/.nvm/versions/node/v20.13.1/bin/node" "/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build" "--update-binary"
node-pre-gyp ERR! cwd /Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas
node-pre-gyp ERR! node -v v20.13.1
node-pre-gyp ERR! node-pre-gyp -v v1.0.11
node-pre-gyp ERR! not ok 
Failed to execute '/Users/jimodonnell/.nvm/versions/node/v20.13.1/bin/node /Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp/bin/node-

error Command failed.
Exit code: 1
Command: /Users/jimodonnell/.nvm/versions/node/v20.13.1/bin/node
Arguments: /Users/jimodonnell/zooniverse/front-end-monorepo/.yarn/releases/yarn-1.22.22.cjs install --frozen-lockfile
Directory: /Users/jimodonnell/zooniverse/front-end-monorepo/packages/lib-react-components
Output:

To Reproduce

yarn bootstrap and the build fails during the yarn install --frozen-lockfile step.

I'm not sure if there's something specific to my build environment that breaks the install just for me. node-canvas has to be compiled from source on Apple CPUs.

Additional context

To work around this, install with install scripts disabled. This is good security practice anyway, to guard against supply-chain attacks that install malware on your laptop from NPM.

yarn install --ignore-scripts --frozen-lockfile
yarn bootstrap

I found this in the node-canvas Readme:

By default, pre-built binaries will be downloaded if you're on one of the following platforms:

  • macOS x86/64 (not Apple silicon)
  • Linux x86/64 (glibc only)
  • Windows x86/64

I'm trying to bootstrap the monorepo on an M3 Macbook, so that explains why it's trying to compile node-canvas from source.

eatyourgreens commented 3 days ago

I think #6349 introduced this.

eatyourgreens commented 3 days ago

To be honest, the subject viewer is never rendered on the server, so node-canvas isn't really needed in the build. On a personal level, I'm also not sure about the ethics of supporting Automattic at the moment.

goplayoutside3 commented 3 days ago

@eatyourgreens see instructions for MacOS installation of canvas in lib-subject-viewer's Readme and the node-canvas issue.

eatyourgreens commented 3 days ago

Thanks! The brew install shouldn't be necessary, because those Node binaries are never used.

eatyourgreens commented 3 days ago

--ignore-scripts fixed the build for me. I don't think any of the post-install scripts are needed on MacOS.

eatyourgreens commented 3 days ago

The Docker builds use --ignore-scripts, and the GitHub workflows too, so I've opened #6393 to add it to the bootstrap scripts. That will fix this by not building the canvas binaries.

Subject viewer tests run fine in Node, even without those binaries. I wasn't expecting that, to be honest.

eatyourgreens commented 3 days ago

After a bit of experimentation, it turns out you can completely remove canvas from the Volumetric Viewer without breaking the tests or the storybook. It isn't used in the browser and the tests aren't using it to render <canvas> in Node either.

TBH, given that the subject viewers are only ever run in browsers, never in Node, I'd test the Volumetric Viewer with Playwright rather than JSDOM. I've been using Playwright for some WebGL/canvas stuff I'm working on, and it just works out of the box. It also allows me to run my tests in Firefox, Chromium and Webkit, which is really useful.