lovell / farmhash

Node.js implementation of FarmHash, Google's family of high performance hash functions
Apache License 2.0
444 stars 33 forks source link

Enhancement: prebuilt binaries for linux-arm64 and linuxmusl-arm64 #48

Open lovell opened 6 months ago

lovell commented 6 months ago

Future possible enhancement to provide prebuilt binaries for Linux (glibc and musl) running on ARM64.

CircleCI can be used for this purpose e.g. https://github.com/lovell/sharp/blob/main/.circleci/config.yml


I don't have any immediate plans to work on this particular task so if anyone requires this urgently then I am available for paid consulting. Anyone wishing to support open source software development by purchasing 1 day of my time to work on this task is very welcome to do so via https://buy.stripe.com/6oE00r1Qe7sgeli6oR - thanks in advance.

wodCZ commented 5 months ago

Please do 🙏

Got farmhash@3.3.1 as a new dependency of firebase-admin@12.1.0, and my build on alpine docker on Apple Silicon (M2) fails because of missing python.

Getting this prebuild-install warning:

No prebuilt binaries found (target=3 runtime=napi arch=arm64 libc=musl platform=linux)
busybox11 commented 5 months ago

You're getting this because you're using Docker. Native Apple Silicon prebuilt binary exists. You can either bundle build tools in your image for Linux or straight up use MacOS instead. (for the time being, of course)

wodCZ commented 5 months ago

@busybox11 Thanks for the response 🙏

I use Docker in production (built & running on linux amd64, working just fine there 👍). I just can't build the image locally to verify it works, which is not a blocker.

I suppose I could also install python and other dependencies in the image as a temporary workaround, but again, it builds fine on target environment, so that would be a waste. I personally can just wait it out until someone knowledgeable and kind adds the prebuilt binaries. I guess I just wanted to let the OP know that the library is now used in firebase-admin (1.5M downloads/wk) and eventually more users with setup similar to mine will show up in this issue.

Use Docker they said. No more "works on my machine" they said. 🤷‍♂️

busybox11 commented 5 months ago

Yes, this is what i meant! I also had to add C++ / Python build tools in my Docker image to build my Node project. But now it works flawlessly on my ARM CI server. And I also have a cache of the built deps so that I don't have to rebuild every time, I only lose like 2-3 seconds per build compared to before. It's not a big deal, although a little annoying indeed.

To be fair, it's not Docker's fault! It works as expected in this case, it doesn't work on Linux ARM, it won't work on any other Linux ARM out of the box either haha. At least it makes it replicable in a barebones environment like an Alpine image.

Now that I think of it you could run the Docker build by specifying the amd64 architecture, which would probably 1:1 replicate your production environment. Just use the --platform linux/amd64 CLI flag when building. The error should disappear since it would use Rosetta and the prebuilt binary for x86 which already exists. A step further in making sure your Docker image works as expected on x86 as well!

lovell commented 5 months ago

I guess I just wanted to let the OP know that the library is now used in firebase-admin (1.5M downloads/wk) and eventually more users with setup similar to mine will show up in this issue.

Yes, this was unexpected, and I created this issue in response to learning of this new dependee.

adarshmadrecha commented 5 months ago

I also landed here because my npm i command failed on ARM based Ubuntu 22.04 on aws. after upgrading from firebase-admin 12.0.0 to 12.1.0 Please add pre built binaries.

lhermann commented 5 months ago

In case others are looking for the same issue: In my case the node:18-alpine image produced the error while the node:18 image worked.

busybox11 commented 5 months ago

@lhermann The regular node image works because it comes bundled with Python and other build tools that this project requires.

It is not using prebuilt binaries, which will have a hit on Docker image build speed and performance use - which are important when billed by usage, on GH Actions for instance.

Although yes, to be fair, you won't have anything special to do on the regular node image to use this dep!