search

iotaledger / iota-sdk

The IOTA SDK provides developers with a seamless experience to develop on IOTA by providing account abstractions and clients to interact with node APIs.
Apache License 2.0
58 stars 43 forks source link

Support arm64 architectures for the nodeJS bindings #97

Open Tuditi opened 2 years ago

Tuditi commented 2 years ago

Description

We are missing binaries that support the arm64 architecture

Motivation

prebuild only builds binaries targetting the x64 architecture. Since a lot of users use arm64, we need to update the workflows to include builds for arm64 as well.

Are you planning to do it yourself in a pull request?

Yes

rajivshah3 commented 2 years ago

Similar work was done for Java here: https://github.com/iotaledger/wallet.rs/blob/develop/.github/workflows/java_binding_publish.yml

Since GitHub Actions does not have arm64-based Mac runners, we'll have to cross-compile on the x64 runners. I also wonder if setting the macOS deployment target to 10.12 will pose any issues here 🤔 If necessary we can simply move that to only be set when building for x64

https://github.com/iotaledger/wallet.rs/blob/561dd5886c0736ef8cd0b8c400a9f4797f85a81a/.github/workflows/covector-version-or-publish.yml#L101

Tuditi commented 1 year ago

We reverted the change, since it wasn't building properly on M1 chips: https://github.com/iotaledger/wallet.rs/pull/1779

Thoralf-M commented 1 year ago

For macos I got it working now, ubuntu and windows still have errors: ubuntu:

Compiling ring v0.16.20
error: failed to run custom build command for `ring v0.16.20`

Caused by:
  process didn't exit successfully: `/home/runner/work/iota-sdk/iota-sdk/target/release/build/ring-665604c6fe378487/build-script-build` (exit status: 101)
  --- stdout
  OPT_LEVEL = Some("3")
  TARGET = Some("aarch64-unknown-linux-gnu")
  HOST = Some("x86_64-unknown-linux-gnu")
  cargo:rerun-if-env-changed=CC_aarch64-unknown-linux-gnu
  CC_aarch64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_aarch64_unknown_linux_gnu
  CC_aarch64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=TARGET_CC
  TARGET_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  RUSTC_LINKER = None
  cargo:rerun-if-env-changed=CROSS_COMPILE
  CROSS_COMPILE = None
  cargo:rerun-if-env-changed=CFLAGS_aarch64-unknown-linux-gnu
  CFLAGS_aarch64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_aarch64_unknown_linux_gnu
  CFLAGS_aarch64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=TARGET_CFLAGS
  TARGET_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("false")
  CARGO_CFG_TARGET_FEATURE = Some("neon")

  --- stderr
  running "aarch64-linux-gnu-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-DNDEBUG" "-c" "-o/home/runner/work/iota-sdk/iota-sdk/target/aarch64-unknown-linux-gnu/release/build/ring-3b981594b08[71](https://github.com/Thoralf-M/iota-sdk/actions/runs/5457115380/jobs/9930784844#step:24:72)2da/out/aesv8-armx-linux64.o" "/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.16.20/pregenerated/aesv8-armx-linux64.S"
  thread 'main' panicked at 'failed to execute ["aarch64-linux-gnu-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-DNDEBUG" "-c" "-o/home/runner/work/iota-sdk/iota-sdk/target/aarch64-unknown-linux-gnu/release/build/ring-3b9[81](https://github.com/Thoralf-M/iota-sdk/actions/runs/5457115380/jobs/9930784844#step:24:82)594b0[87](https://github.com/Thoralf-M/iota-sdk/actions/runs/5457115380/jobs/9930784844#step:24:88)12da/out/aesv8-armx-linux64.o" "/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.16.20/pregenerated/aesv8-armx-linux64.S"]: No such file or directory (os error 2)', /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ring-0.16.20/build.rs:653:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
Did not copy "cdylib:iota-sdk-nodejs"
prebuild ERR! build Error: cargo-cp-artifact -ac iota-sdk-nodejs ./index.node -- cargo build --release --message-format=json-render-diagnostics --target aarch64-unknown-linux-gnu  failed with exit code 101
prebuild ERR! build     at exports.spawnFailed (/home/runner/work/iota-sdk/iota-sdk/bindings/nodejs/node_modules/prebuild/error.js:10:10)
prebuild ERR! build     at ChildProcess.<anonymous> (/home/runner/work/iota-sdk/iota-sdk/bindings/nodejs/node_modules/prebuild/util.js:36:14)
prebuild ERR! build     at ChildProcess.emit (node:events:513:28)
prebuild ERR! build     at ChildProcess._handle.onexit (node:internal/child_process:2[91](https://github.com/Thoralf-M/iota-sdk/actions/runs/5457115380/jobs/9930784844#step:24:92):12)
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Error: Process completed with exit code 2.

windows:

$ prebuild --runtime napi --target 6 --prepack 'cargo-cp-artifact -ac iota-sdk-nodejs ./index.node -- cargo build --release --message-format=json-render-diagnostics --target aarch64-pc-windows-msvc' --strip --arch arm64
prebuild info begin Prebuild version 11.0.4
prebuild info build Preparing to prebuild @iota/sdk@1.0.0-rc.1 for napi 6 on win32-true using node-gyp
prebuild info find Python using Python version 3.10.11 found at "C:\hostedtoolcache\windows\Python\3.10.11\x64\python3.exe"
prebuild info find VS using VS2019 (16.11.33801.447) found at:
prebuild info find VS "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise"
prebuild info find VS run with --verbose for detailed information
(node:5564) [DEP0150] DeprecationWarning: Setting process.config is deprecated. In the future the property will be read-only.
(Use `node --trace-deprecation ...` to show where the warning was created)
prebuild info spawn C:\hostedtoolcache\windows\Python\3.10.11\x64\python3.exe
prebuild info spawn args [
prebuild info spawn args   'D:\\a\\iota-sdk\\iota-sdk\\bindings\\nodejs\\node_modules\\node-gyp\\gyp\\gyp_main.py',
prebuild info spawn args   'binding.gyp',
prebuild info spawn args   '-f',
prebuild info spawn args   'msvs',
prebuild info spawn args   '-I',
prebuild info spawn args   'D:\\a\\iota-sdk\\iota-sdk\\bindings\\nodejs\\build\\config.gypi',
prebuild info spawn args   '-I',
prebuild info spawn args   'D:\\a\\iota-sdk\\iota-sdk\\bindings\\nodejs\\node_modules\\node-gyp\\addon.gypi',
prebuild info spawn args   '-I',
prebuild info spawn args   'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\prebuild\\napi\\[18](https://github.com/Thoralf-M/iota-sdk/actions/runs/5457115380/jobs/9930785075#step:25:19).16.1\\include\\node\\common.gypi',
prebuild info spawn args   '-Dlibrary=shared_library',
prebuild info spawn args   '-Dvisibility=default',
prebuild info spawn args   '-Dnode_root_dir=C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\prebuild\\napi\\18.16.1',
prebuild info spawn args   '-Dnode_gyp_dir=D:\\a\\iota-sdk\\iota-sdk\\bindings\\nodejs\\node_modules\\node-gyp',
prebuild info spawn args   '-Dnode_lib_file=C:\\\\Users\\\\RUNNER~1\\\\AppData\\\\Local\\\\Temp\\\\prebuild\\\\napi\\\\18.16.1\\\\<(target_arch)\\\\node.lib',
prebuild info spawn args   '-Dmodule_root_dir=D:\\a\\iota-sdk\\iota-sdk\\bindings\\nodejs',
prebuild info spawn args   '-Dnode_engine=v8',
prebuild info spawn args   '--depth=.',
prebuild info spawn args   '--no-parallel',
prebuild info spawn args   '--generator-output',
prebuild info spawn args   'D:\\a\\iota-sdk\\iota-sdk\\bindings\\nodejs\\build',
prebuild info spawn args   '-Goutput_dir=.'
prebuild info spawn args ]
prebuild info spawn C:\Program Files (x86)\Microsoft Visual Studio\20[19](https://github.com/Thoralf-M/iota-sdk/actions/runs/5457115380/jobs/9930785075#step:25:20)\Enterprise\MSBuild\Current\Bin\MSBuild.exe
prebuild info spawn args [
prebuild info spawn args   'build/binding.sln',
prebuild info spawn args   '/clp:Verbosity=minimal',
prebuild info spawn args   '/nologo',
prebuild info spawn args   '/p:Configuration=Release;Platform=Win32'
prebuild info spawn args ]
Building the projects in this solution one at a time. To enable parallel build, please add the "-m" switch.
  win_delay_load_hook.cc
LINK : fatal error LNK1104: cannot open file 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\prebuild\\napi\\18.16.1\\true\\node.lib' [D:\a\iota-sdk\iota-sdk\bindings\nodejs\build\index.vcxproj]
prebuild ERR! build error 
prebuild ERR! stack Error: `C:\Program Files (x86)\Microsoft Visual Studio\[20](https://github.com/Thoralf-M/iota-sdk/actions/runs/5457115380/jobs/9930785075#step:25:21)19\Enterprise\MSBuild\Current\Bin\MSBuild.exe` failed with exit code: 1
prebuild ERR! stack     at ChildProcess.onExit (D:\a\iota-sdk\iota-sdk\bindings\nodejs\node_modules\node-gyp\lib\build.js:194:[23](https://github.com/Thoralf-M/iota-sdk/actions/runs/5457115380/jobs/9930785075#step:25:24))
prebuild ERR! stack     at ChildProcess.emit (node:events:513:[28](https://github.com/Thoralf-M/iota-sdk/actions/runs/5457115380/jobs/9930785075#step:25:29))
prebuild ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:[29](https://github.com/Thoralf-M/iota-sdk/actions/runs/5457115380/jobs/9930785075#step:25:30)1:12)
prebuild ERR! not ok 
prebuild ERR! build Error: `C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\MSBuild.exe` failed with exit code: 1
prebuild ERR! build     at ChildProcess.onExit (D:\a\iota-sdk\iota-sdk\bindings\nodejs\node_modules\node-gyp\lib\build.js:194:23)
prebuild ERR! build     at ChildProcess.emit (node:events:[51](https://github.com/Thoralf-M/iota-sdk/actions/runs/5457115380/jobs/9930785075#step:25:52)3:28)
prebuild ERR! build     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
error Command failed with exit code 2.
Tuditi commented 1 year ago

The true value in the path below seems rather odd to me.

LINK : fatal error LNK1104: cannot open file 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\prebuild\\napi\\18.16.1\\true\\node.lib' [D:\a\iota-sdk\iota-sdk\bindings\nodejs\build\index.vcxproj]```

Maybe the target arch is not passed correctly? 

prebuild info spawn args   '-Dnode_lib_file=C:\\\\Users\\\\RUNNER~1\\\\AppData\\\\Local\\\\Temp\\\\prebuild\\\\napi\\\\18.16.1\\\\<(target_arch)\\\\node.lib',
thibault-martinez commented 1 year ago

Macos arm64 has been addressed. Linux and windows will be done at a later stage as to not block the release. Moving it to the 1.1 milestone.

andy-uphold commented 4 months ago

Any progress on this? I'm having linux issues.

andy-uphold commented 3 months ago

Still having issues. Anyone working on this?