search

yarnpkg / yarn

The 1.x line is frozen - features and bugfixes now happen on https://github.com/yarnpkg/berry
https://classic.yarnpkg.com
Other
41.45k stars 2.73k forks source link

Support --arch and --target_arch to build modules for other architectures #2221

Open nsgundy opened 7 years ago

nsgundy commented 7 years ago

Do you want to request a feature or report a bug? Feature

What is the current behavior? Not able to pass parameters so that yarn can be told to build packages for a specific architecture. Right now, in order to build for a specific target other than the architecture of the machine yarn is executed on, one has to run (example given to compile for arm):

yarn --production --ignore-scripts
npm rebuild --arch=arm --target_arch=arm

What is the expected behavior? Ability to pass --arch and --target_arch that yarn would then pass on to node-gyp, so that the above example would turn into:

yarn --production --arch=arm --target_arch=arm

Please mention your node.js, yarn and operating system version. node v6.9.1 yarn 0.17.10 Ubuntu 14.04

duncanfinney commented 7 years ago

+1

bestander commented 7 years ago

PR is welcome

SteveDesmond-ca commented 6 years ago

This should be resolved now. Cross-compiling is available by setting the npm_config_arch environment variable.

Globegitter commented 5 years ago

That aside it would also be great to support --target_platform, e.g. --target_platform=linux to be able for example to natively build a docker container on osx. Or is it possible to set a env variable for that also?

SteveDesmond-ca commented 5 years ago

https://docs.npmjs.com/misc/config theoretically if npm has a flag for it, you can set the env var the same way, e.g. something like npm_config_platform=linux

muratcorlu commented 3 years ago

Can someone explain is this still an issue? If not, how can we set arch for installation?

node-sass installations on M1 Macs are a problem now. And I couldn't figure out how to fix this with Yarn. npm workaround shared here works well: https://github.com/sass/node-sass/issues/3033#issuecomment-763180778

AndrewMuise commented 3 years ago

@muratcorlu as mentioned above, set npm_config_arch environment variable for your target arch when installing your node_modules. For your case of M1 this would be arm64.

muratcorlu commented 3 years ago

@AndrewMuise No, actually I'm trying to install node-sass with x64 target since it doesn't build with arm64 for now. With npm, this works:

npm install --target_arch=x64

I tried to run NPM_CONFIG_ARCH=x64 yarn command but didn't work. Am I missing something?

SteveDesmond-ca commented 3 years ago

It's case sensitive, all lowercase, I know that doesn't follow env var convention but I think it was so args would match better

muratcorlu commented 3 years ago

I couldn't make it. Both npm_config_arch=x64 npm i and npm_config_arch=x64 yarn doesn't affect anything.

I try with Node 16 on an M1 Mac Mini with Big Sur.

Can you give an official document link about this env variable? I don't see an arch config parameter here: https://docs.npmjs.com/cli/v7/using-npm/config

Eli-Black-Work commented 3 years ago

@muratcorlu, From the Windows command line, I'm doing this:

SET "npm_config_arch=xxxxx" && SET "npm_config_platform=xxxxx" && yarn install

That seems to work when I'm installing electron, at least.

Not sure how to set environment variables on Mac command line, but I think the concept should be similar 🙂

uasi commented 3 years ago

@muratcorlu npm_config_arch env is recognized by node-gyp. See https://github.com/nodejs/node-gyp/blob/master/README.md#configuration

Also, there is npm_config_target_arch that is recognized by node-pre-gyp.

h-grig commented 2 years ago

Here are 2 options for npm and yarn to specify arch, platform and libc

npm install --target_arch=arm64 --target_platform=linux --target_libc=glibc
npm_config_target_arch=arm64 npm_config_target_platform=linux npm_config_target_libc=glibc yarn instal
dkwo commented 1 year ago

@h-grig is there a similar option for parcel? right now, cross-compiling from x86_64-musl to aarch64 fails for a project that uses first node-gyp and then parcel:

=> chronograf-1.10.1_1: running pre_build ...
cd ui && yarn --no-progress --no-emoji
yarn install v1.22.18
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning "workspace-aggregator-8ca6dd9e-a42f-4d40-8c1f-d1d6b4e21f81 > chronograf-ui > cypress-image-snapshot@4.0.1" has incorrect peer dependency "cypress@^4.5.0".
warning "workspace-aggregator-8ca6dd9e-a42f-4d40-8c1f-d1d6b4e21f81 > chronograf-ui > react-tooltip@3.11.6" has incorrect peer dependency "react@>=^16.0.0".
warning "workspace-aggregator-8ca6dd9e-a42f-4d40-8c1f-d1d6b4e21f81 > chronograf-ui > react-tooltip@3.11.6" has incorrect peer dependency "react-dom@>=^16.0.0".
warning "workspace-aggregator-8ca6dd9e-a42f-4d40-8c1f-d1d6b4e21f81 > chronograf-ui > cypress-image-snapshot > jest-image-snapshot@4.2.0" has incorrect peer dependency "jest@>=20 <=26".
[4/4] Building fresh packages...
$ echo "" > ../node_modules/go.mod
Done in 54.08s.
cd ui && yarn run clean && yarn run build
yarn run v1.22.18
$ rm -rf ./build/* && rm -rf ./.cache ../.parcel-cache
Done in 0.12s.
yarn run v1.22.18
$ APP_VERSION=$npm_package_version GIT_SHA=$(git rev-parse HEAD) parcel build --log-level error
fatal: not a git repository (or any of the parent directories): .git
Error: Error loading shared library ld-linux-aarch64.so.1: No such file or directory (needed by /builddir/chronograf-1.10.1/node_modules/@parcel/watcher/build/Release/watcher.node)
    at Object.Module._extensions..node (node:internal/modules/cjs/loader:1249:18)
    at Module.load (node:internal/modules/cjs/loader:1043:32)
    at Function.Module._load (node:internal/modules/cjs/loader:878:12)
    at Module.require (node:internal/modules/cjs/loader:1067:19)
    at require (/builddir/chronograf-1.10.1/node_modules/v8-compile-cache/v8-compile-cache.js:159:20)
    at load (/builddir/chronograf-1.10.1/node_modules/node-gyp-build/index.js:22:10)
    at Object.<anonymous> (/builddir/chronograf-1.10.1/node_modules/@parcel/watcher/index.js:1:104)
    at Module._compile (/builddir/chronograf-1.10.1/node_modules/v8-compile-cache/v8-compile-cache.js:192:30)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1219:10)
    at Module.load (node:internal/modules/cjs/loader:1043:32) {
  code: 'ERR_DLOPEN_FAILED'
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
make: *** [Makefile:62: .jssrc] Error 1
=> ERROR: chronograf-1.10.1_1: pre_build: 'npm_config_arch=arm64 CFLAGS="$CFLAGS -fPIC" CXXFLAGS="$CXXFLAGS -fPIC" make .jsdep assets' exited with 2
zN3utr4l commented 6 months ago

@muratcorlu, From the Windows command line, I'm doing this:

SET "npm_config_arch=xxxxx" && SET "npm_config_platform=xxxxx" && yarn install

That seems to work when I'm installing electron, at least.

Not sure how to set environment variables on Mac command line, but I think the concept should be similar 🙂

It doesn't work I think, if I use --inline-builds, it seems to always build from the current system:

PS G:\Test\test\server> SET "npm_config_arch=ia32" ; SET "npm_config_target_arch=ia32" ; SET "npm_config_platform=linux" ; yarn install --inline-builds
➤ YN0000: · Yarn 4.2.1
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed in 0s 284ms
➤ YN0000: ┌ Post-resolution validation
➤ YN0002: │ test@workspace:. doesn't provide eslint (p5f692), requested by eslint-config-prettier.
➤ YN0086: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements <hash> for details, where <hash> is the six-letter p-prefixed code.
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed in 0s 708ms
➤ YN0000: ┌ Link step
➤ YN0007: │ canvas@npm:2.11.2 must be built because it never has been before or the last one failed
➤ YN0007: │ sqlite3@npm:5.1.7 [ce7f3] must be built because it never has been before or the last one failed
➤ YN0007: │ @serialport/bindings-cpp@npm:12.0.1 must be built because it never has been before or the last one failed
➤ YN0007: │ es5-ext@npm:0.10.62 must be built because it never has been before or the last one failed
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info it worked if it ends with ok
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info using node-pre-gyp@1.0.11
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info using node@18.20.2 | win32 | x64
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp http GET https://github.com/Automattic/node-canvas/releases/download/v2.11.2/canvas-v2.11.2-node-v108-win32-unknown-x64.tar.gz
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/canvas.exp
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/canvas.iobj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/canvas.ipdb
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/canvas.lib
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/canvas.node
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/canvas.pdb
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libbrotlicommon.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libbrotlidec.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libbz2-1.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libcairo-2.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libcairo-gobject-2.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libdatrie-1.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libexpat-1.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libffi-8.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libfontconfig-1.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libfreetype-6.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libfribidi-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libgcc_s_seh-1.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libgdk_pixbuf-2.0-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libgif-7.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libgio-2.0-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libglib-2.0-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libgmodule-2.0-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libgobject-2.0-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libgraphite2.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libharfbuzz-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libiconv-2.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libintl-8.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libjpeg-8.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/liblzma-5.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libpango-1.0-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libpangocairo-1.0-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libpangoft2-1.0-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libpangowin32-1.0-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libpcre2-8-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libpixman-1-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libpng16-16.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/librsvg-2-2.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libstdc++-6.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libthai-0.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libwinpthread-1.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/libxml2-2.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/canvas.node.recipe
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/canvas.tlog/
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/canvas.tlog/canvas.lastbuildstate
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/canvas.tlog/canvas.write.1u.tlog
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/canvas.tlog/CL.command.1.tlog
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/canvas.tlog/CL.read.1.tlog
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/canvas.tlog/CL.write.1.tlog
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/canvas.tlog/link.command.1.tlog
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/canvas.tlog/link.read.1.tlog
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/canvas.tlog/link.write.1.tlog
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/backend/
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/backend/Backend.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/backend/ImageBackend.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/backend/PdfBackend.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/backend/SvgBackend.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/Backends.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/bmp/
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/bmp/BMPParser.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/Canvas.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/CanvasGradient.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/CanvasPattern.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/CanvasRenderingContext2d.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/closure.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/color.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/Image.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/ImageData.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/init.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/src/register_font.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/obj/canvas/win_delay_load_hook.obj
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info install unpacking Release/zlib1.dll
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info extracted file count: 76 
➤ YN0000: │ canvas@npm:2.11.2 STDOUT [canvas] Success: "G:\Test\test\server\node_modules\canvas\build\Release\canvas.node" is installed via remote
➤ YN0000: │ canvas@npm:2.11.2 STDERR node-pre-gyp info ok
➤ YN0000: └ Completed in 15s 249ms
➤ YN0000: · Done with warnings in 16s 370ms
chetbox commented 1 week ago

I found that you can install packages for multiple architectures or platforms using the supportedArchitectures option in .yarnrc.yml:

supportedArchitectures:
  cpu: ['x64', 'arm64']

Then just run yarn.

In my case, instead of running yarn --arch=x64 like I did with Yarn 1, I now with Yarn 4 I run:

echo -e "supportedArchitectures:\n  cpu: ['x64']" >> .yarnrc.yml
yarn