arm64 Linux prebuilts

[AAGaming00]

Could prebuilts be added for arm64 Linux? I'm wasting around ten minutes in ci right now waiting for this to build each time.

happy to help with configuring ci and whatever to accomplish this

[uhop]

I am restricted by free runners provided by Github. IIRC they don't provide arm64 Linux runners. I add arm64 MacOS pre-compiled extensions running them manually on my local computer. Unfortunately, I cannot accept 3rd-party builds for security reasons.

One possible solution for you is to use a private mirror, which is described in https://github.com/uhop/node-re2/wiki/Precompiled-versions#private-mirrors

[AAGaming00]

I am restricted by free runners provided by Github. IIRC they don't provide arm64 Linux runners. I add arm64 MacOS pre-compiled extensions running them manually on my local computer. Unfortunately, I cannot accept 3rd-party builds for security reasons.

One possible solution for you is to use a private mirror, which is described in https://github.com/uhop/node-re2/wiki/Precompiled-versions#private-mirrors

could build them on the free runners using qemu static potentially with something like docker buildx, or some kind of cross compiling

as for arm Mac GitHub actually offers arm Mac runners as of a few weeks ago

[uhop]

Sounds simple, even trivial. When can I expect a PR?

[AAGaming00]

Have a lot of stuff going on right now, so no ETA, but I'll definitely work on it and send a PR when I have a chance to!

[rynop]

I can't get it to successfully build on arm64 linux (AWS Graviton).

167 error gyp info it worked if it ends with ok
167 error gyp info using node-gyp@10.0.1
167 error gyp info using node@16.15.0 | linux | arm64
167 error gyp info find Python using Python version 3.7.10 found at "/bin/python3"
167 error gyp WARN EACCES current user ("webapp") does not have permission to access the dev dir "/root/.cache/node-gyp/16.15.0"
167 error gyp WARN EACCES attempting to reinstall using temporary dev dir "/tmp/.node-gyp"
167 error gyp info spawn /bin/python3
167 error gyp info spawn args [
167 error gyp info spawn args '/var/app/staging/node_modules/node-gyp/gyp/gyp_main.py',
167 error gyp info spawn args 'binding.gyp',
167 error gyp info spawn args '-f',
167 error gyp info spawn args 'make',
167 error gyp info spawn args '-I',
167 error gyp info spawn args '/var/app/staging/node_modules/re2/build/config.gypi',
167 error gyp info spawn args '-I',
167 error gyp info spawn args '/var/app/staging/node_modules/node-gyp/addon.gypi',
167 error gyp info spawn args '-I',
167 error gyp info spawn args '/tmp/.node-gyp/16.15.0/include/node/common.gypi',
167 error gyp info spawn args '-Dlibrary=shared_library',
167 error gyp info spawn args '-Dvisibility=default',
167 error gyp info spawn args '-Dnode_root_dir=/tmp/.node-gyp/16.15.0',
167 error gyp info spawn args '-Dnode_gyp_dir=/var/app/staging/node_modules/node-gyp',
167 error gyp info spawn args '-Dnode_lib_file=/tmp/.node-gyp/16.15.0/<(target_arch)/node.lib',
167 error gyp info spawn args '-Dmodule_root_dir=/var/app/staging/node_modules/re2',
167 error gyp info spawn args '-Dnode_engine=v8',
167 error gyp info spawn args '--depth=.',
167 error gyp info spawn args '--no-parallel',
167 error gyp info spawn args '--generator-output',
167 error gyp info spawn args 'build',
167 error gyp info spawn args '-Goutput_dir=.'
167 error gyp info spawn args ]
167 error gyp info spawn make
167 error gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
167 error g++: error: unrecognized command line option '-std=c++2a'; did you mean '-std=c++03'?
167 error make: *** [Release/obj.target/re2/lib/addon.o] Error 1

@AAGaming00 did you have to do anything special (like install some os deps 1st) to get it to build via gyp?

[rynop]

n/m re2 requires c++ 20 https://github.com/uhop/node-re2/issues/174#issuecomment-1593632228

+1 to prebuilt linux binary :) AWS graviton instances are pretty prevalent, they are arm64

[AAGaming00]

I can't get it to successfully build on arm64 linux (AWS Graviton).

167 error gyp info it worked if it ends with ok
167 error gyp info using node-gyp@10.0.1
167 error gyp info using node@16.15.0 | linux | arm64
167 error gyp info find Python using Python version 3.7.10 found at "/bin/python3"
167 error gyp WARN EACCES current user ("webapp") does not have permission to access the dev dir "/root/.cache/node-gyp/16.15.0"
167 error gyp WARN EACCES attempting to reinstall using temporary dev dir "/tmp/.node-gyp"
167 error gyp info spawn /bin/python3
167 error gyp info spawn args [
167 error gyp info spawn args '/var/app/staging/node_modules/node-gyp/gyp/gyp_main.py',
167 error gyp info spawn args 'binding.gyp',
167 error gyp info spawn args '-f',
167 error gyp info spawn args 'make',
167 error gyp info spawn args '-I',
167 error gyp info spawn args '/var/app/staging/node_modules/re2/build/config.gypi',
167 error gyp info spawn args '-I',
167 error gyp info spawn args '/var/app/staging/node_modules/node-gyp/addon.gypi',
167 error gyp info spawn args '-I',
167 error gyp info spawn args '/tmp/.node-gyp/16.15.0/include/node/common.gypi',
167 error gyp info spawn args '-Dlibrary=shared_library',
167 error gyp info spawn args '-Dvisibility=default',
167 error gyp info spawn args '-Dnode_root_dir=/tmp/.node-gyp/16.15.0',
167 error gyp info spawn args '-Dnode_gyp_dir=/var/app/staging/node_modules/node-gyp',
167 error gyp info spawn args '-Dnode_lib_file=/tmp/.node-gyp/16.15.0/<(target_arch)/node.lib',
167 error gyp info spawn args '-Dmodule_root_dir=/var/app/staging/node_modules/re2',
167 error gyp info spawn args '-Dnode_engine=v8',
167 error gyp info spawn args '--depth=.',
167 error gyp info spawn args '--no-parallel',
167 error gyp info spawn args '--generator-output',
167 error gyp info spawn args 'build',
167 error gyp info spawn args '-Goutput_dir=.'
167 error gyp info spawn args ]
167 error gyp info spawn make
167 error gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
167 error g++: error: unrecognized command line option '-std=c++2a'; did you mean '-std=c++03'?
167 error make: *** [Release/obj.target/re2/lib/addon.o] Error 1

@AAGaming00 did you have to do anything special (like install some os deps 1st) to get it to build via gyp?

distro issue, compiler too old but it seems like you already realized that

also set JOBS=max if you don't want the build to take forever

[viceice]

I successfully cross compile it on github action hosted ubuntu runner 💪

https://github.com/containerbase/node-re2-prebuild/actions/runs/7062462947/job/19226323007

[viceice]

@uhop Are you open to a PR which cross compiles the linux-arm64 binaries ?

[uhop]

I am open to PRs.

I had a cursory look at how it was done and I applaud you that all correct incantations were found and tested. Major kudos! But unless I missed something obvious, it looks like it can be simplified. For example, you build re2 with NPM using cross-platform compilers and setting a different architecture, and then the Node docker image for that architecture is spun off to test the result. Wouldn't be easier to spin off the same Node image and do the standard build on that architecture, testing and saving it as in the main build action?

[AAGaming00]

Wouldn't be easier to spin off the same Node image and do the standard build on that architecture, testing and saving it as in the main build action?

The issue here is that would result in the native aarch64 compilers running under qemu-user, which, especially on the fairly limited resource runners GitHub provides, is extremely slow. Even on my 8 core laptop, running a build under qemu even with JOBS=max takes 4 minutes.

Cross compiling is orders of magnitude faster.

[AAGaming00]

@viceice you may want to consider setting JOBS=max in your workflow if you don't already, it should help speed it up a bit.

[viceice]

Wouldn't be easier to spin off the same Node image and do the standard build on that architecture, testing and saving it as in the main build action?

The issue here is that would result in the native aarch64 compilers running under qemu-user, which, especially on the fairly limited resource runners GitHub provides, is extremely slow. Even on my 8 core laptop, running a build under qemu even with JOBS=max takes 4 minutes.

Cross compiling is orders of magnitude faster.

that's why I use cross compile 👍 will try the jobs environment variable

[uhop]

@viceice got it.

TBH, I release new versions relatively rarely, so I don't care how long it takes to compile a version. 4 minutes are acceptable time. I'll go for simplicity, reuse, compartmentalization, and a smaller number of moving parts. If tomorrow I update a publishing process I don't want to duplicate it.

OTOH, if it interferes with GitHub Actions in any negative way, or increases the failure rate (builds fail for no apparent reasons even now), I would like to have an alternative you already provided.

Hmm...

[AAGaming00]

4 minutes is an acceptable time

Building on a 16 thread ryzen 7 machine taking 4 minutes means the small GitHub actions runner will take much, much longer.

[uhop]

TBH, 40 min, or even 4 hours, would be acceptable for infrequent builds… It would be nice to have both variants, so we can choose.

[bradleyayers]

I'm not sure what the latest thinking is on this issue, but I'd be happy to spend a few hours trying to move this forward if that would help…

edit: just saw https://resources.github.com/devops/accelerate-your-cicd-with-arm-and-gpu-runners-in-github-actions/, perhaps you can join the early list?

[cmanou]

@uhop I ended up doing the qemu approach as a temporary work around till the arm runners exist and it gives you something to compare the cross compilation approach to.

[AAGaming00]

Thanks for resolving this @cmanou! 🎉

[viceice]

Any chance to get this released, so we can deprecate our custom prebuilds?