Closed cswendrowski closed 1 year ago
We build using dockcross docker images that are unversioned and we don't actively test Raspberry Pi, so it's very well possible that we inadvertently broke it.
I don't have time to look into it. Could you check the glibc versions of older prebuilds (maybe even going back to leveldown
)? And/or check what changed in dockcross, if anything. For context, we use prebuildify-cross
to build the binaries.
Note: at some point we do have to end support of some platforms (intentionally) because node 18 moved to glibc 2.28.
It looks like there's at least one temporary easy win, although it will pose a problem for Node 18 - prebuild/docker-images
for prebuildify-cross
offers linux-arm64-lts
, which uses glibc 2.27 for compatibility: https://github.com/prebuild/docker-images/pkgs/container/linux-arm64-lts
Dockcross additionally has the following that should meet the bill (https://github.com/dockcross/dockcross#summary-cross-compilers):
dockcross/linux-armv6-lts
GLIBC 2.28dockcross/linux-armv7-lts
GLIBC 2.28They don't have a prebuildify wrapped verison, however, so changes may need to be submitted to them.
I'll try to figure out getting these builds setup on my machine and start seeing if there's an option to go to an older prebuild, although the right longterm answer initially seems to be getting prebuilds of the above Dockcross images
Note: at some point we do have to end support of some platforms (intentionally) because node 18 moved to glibc 2.28.
I think that's fair (and necessary). Ubuntu 18's 2.27 was just an easy target to test with to debug this issue
prebuild/docker-images for prebuildify-cross offers linux-arm64-lts
Ah right, I forgot (though I was the one who released that).
dockcross/linux-armv6-lts dockcross/linux-armv7-lts
PRs are welcome over at prebuild/docker-images
! I won't be able to review in depth but that's OK seeing as it would be new images (rather than changes to existing images that are in active use).
Hey there @vweevers, I chatted with my team and we'd like to invest some effort into solving this problem, but I wanted to get your input into how to best accomplish the work and verify the changes work.
Step 1) It seems pretty straightforward to update https://github.com/Level/classic-level/blob/main/package.json#L21 to use linux-arm64-lts
. I can do this locally and build on my ARM64 laptop to verify classic level runs as expected with that option, then open a PR. I don't think this requires any updates to prebuildify-cross
, but let me know if I'm wrong
Step 2) Looking at the changes between https://github.com/prebuild/docker-images/tree/master/linux-arm64 and https://github.com/prebuild/docker-images/tree/master/linux-arm64-lts, it seems like the only thing I'd need to do would be to make copies of the armv6
and armv7
images and have them inherit from the dockcross images identified. What I'm unsure on is how to verify that the changes are correct, and whether prebuildify-cross needs an update to see the new images. Any insights would be helpful
Step 1
That's correct.
Step 2
To verify, build the new docker image(s) locally, then create a container with the classic-level
repo mounted as a volume (or clone the git repo inside the container, but then npm install
might not work for lack of root IIRC). Then run prebuildify -t 8.14.0 --napi --strip
in the container (with classic-level
as working directory). Which should give you a functional prebuild.
Note that prebuildify-cross
merely automates these steps. In theory it also supports local docker images but reportedly that's broken (https://github.com/prebuild/prebuildify-cross/pull/16).
Lastly, prebuildify-cross
does not need an update, because it defaults to pulling images from ghcr.io/prebuild/<image>
where <image
> is an image name provided via the -i
command line flag.
I appreciate your help!
@vweevers I've been running into some issues getting a build made via prebuildify-cross
:
gyp: name 'openssl_fips' is not defined while evaluating condition 'openssl_fips != ""' in /home/node/app/deps/leveldb/leveldb.gyp while loading dependencies of binding.gyp while trying to load binding.gyp
I've tried manually setting openssl_fips
as an environment variable, manually setting it as a variable in binding.gyp
, and updating my libssl-dev
without any success
I even tried via a fork of the current Github action, which failed with the same error: https://github.com/cswendrowski/classic-level/actions/runs/4441111255/jobs/7795775652
Unfortunately the logs of the last successful run on this repo are no longer retained (https://github.com/Level/classic-level/actions/runs/2039662848/jobs/2944228919), so I can't see how it succeeded previously.
Any pointers would be appreciated - as a Windows dev, I don't normally dig into the world of compiling C++ on Linux, but I'm trying my best
EDIT: Well, as always, asking questions is a great way to debug. I was able to get past this issue by adding the following block to binding.gyp
, leveldb.gyp
, and snappy.gyp
:
"variables": {
'openssl_fips': ''
}
This doesn't feel like the right answer, and if it is, we'll still need to get those updated in the upstream repos since two of these are git submodule files, but it at least got a build done on my machine
Doesn't ring a bell, I'll check it later and/or trigger a GHA build here.
Back again with an update after much research:
I was able to wrap the dockcross/linux-armv6-lts
and get a build made, but it turns out that Node 16 is not available on armv6, nor does it seem it ever will be. I can still open a PR for this Dockerfile, but it seems like armv6 support should be dropped.
I was unable to do a successful build using a wrapped dockcross/linux-armv7-lts
due to the following issue: https://github.com/dockcross/dockcross/issues/718
Per https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications, the only Pi that uses armv7 is the Pi 2 Model B v1.1, which we are content to drop support for ourselves. I've left a comment on that dockcross issue in hopes that they one day resolve that issue, but I was unable to solve it myself.
That leaves arm64, which we have successfully verified on newer Pi Models. The only work to support those targets is this PR: https://github.com/Level/classic-level/pull/71
Thanks for your support in debugging this issue, and we look forward to consuming this change when available
The readme lists that this should work on Raspberry Pi, but we've traced down that due to a GLIBC issue, this is not the case
Reproduced on an ARM64 laptop running Ubuntu 18:
Ubuntu 18 ships with GLIBC 2.27, Raspian and Pi OS 10 ships 2.28, and Pi OS 11 with 2.29
On an x64 machine also running Ubuntu 18, the prebundle comes compiled with an older version of GLIBC and runs correctly on 2.27
Any guidance or assistance would be appreciated