Closed christianbundy closed 4 years ago
I put together a PR for ARMv8 support, because I saw this:
error loading sodium bindings: No native build was found for platform=android arch=arm64 runtime=node abi=79 uv=1 armv=8 libc=glibc
But the more I think about it, I'm wondering whether I'm already covered because arch=arm64
? I have no idea what I'm doing, to be honest, so https://github.com/prebuild/docker-images/pull/14 might be a total mistake. Thanks for putting up with me. :heart:
See https://github.com/prebuild/docker-images/pull/14#issuecomment-588635467.
Can you try building from source? It might be a gcc mismatch of some sort.
Just built from source and I'm having the same problem, which is surprising.
I was able to clone Level/level (e5f8e61) and npm i && npm t
worked fine -- but that's Level 6, so maybe that's different?
$ npm ls leveldown
@fraction/oasis@2.13.1 /home/christianbundy/src/fraction/oasis
└─┬ @fraction/flotilla@4.0.0
└─┬ ssb-blobs@1.2.2
└─┬ level@5.0.1
└── leveldown@5.4.1
Here's the full log:
There were some copy and paste problems toward the end, maybe from the escape codes used for colors in various postinstall text, so if anything looks funky that's why. I'm going to try downgrading to the Node.js LTS package and see if that helps at all.
WAIT I'm a big dummy and I was running oasis
(using the globally installed package with the bug) instead of npm start
(which would run the code I literally just compiled). The --build-from-source
breaks my sodium-native and sharp, which needs libvips, but otherwise this seems to be running!
Can you summarize what works and what doesn't?
When running npm install --verbose
with Leveldown as a dependency, it seems that the node-gyp-build
command prints the "dlopen failed" error and then tries compiling from source. If I have all of the build dependencies, it compiles from source, but if I don't have the build dependencies then it leaves me with a Leveldown that crashes with the "dlopen failed" error when I start my project. EDIT: Another way to trigger this error is to ^C during the build from source. The logs don't suggest to me that it's using a prebuild, but maybe I don't know what to look for.
My new confusion is that if I try to npm install --verbose --only=prod
in the Leveldown repository, it tries to build from source and then fails when port_posix.h
tries to include Snappy and I get "'snappy.h' file not found". This only seems to happen when building from the repository, which gives me the impression that when using Leveldown as a dependency I have a mixture of prebuilds and building from source, but maybe there's something else going on.
Note: in the interests of sanity, I've switched to Node.js LTS (12), but I've also cleaned my npm cache and removed all of my node_modules
directories so I don't expect it to bite us in the ass while debugging.
When running
npm install --verbose
with Leveldown as a dependency, it seems that thenode-gyp-build
command prints the "dlopen failed" error and then tries compiling from source. If I have all of the build dependencies, it compiles from source, but if I don't have the build dependencies then it leaves me with a Leveldown that crashes with the "dlopen failed" error when I start my project.
That is as expected: node-gyp-build
checks if a suitable prebuild exists and then tries to load it (as a smoke test, if you will). If that fails, it falls back to compiling. But if compiling also fails, then you still only have the (broken) prebuild.
My new confusion is that if I try to
npm install --verbose --only=prod
in the Leveldown repository, it tries to build from source and then fails whenport_posix.h
tries to include Snappy and I get "'snappy.h' file not found".
Snappy is a git submodule. You may have to git submodule update --init --recursive
.
As for why the prebuild is broken, let's find out what _Unwind_Resume
is for and where it is expected to be found. libgcc source code says this:
/* Resume propagation of an existing exception. This is used after
e.g. executing cleanup code, and not to implement rethrowing. */
extern void LIBGCC2_UNWIND_ATTRIBUTE
_Unwind_Resume (struct _Unwind_Exception *);
We disable exceptions though, so I would not expect _Unwind_Resume
to be needed:
https://github.com/Level/leveldown/blob/3e1177dd82de45871c144b1f39ea7b4dfe96e4b1/binding.gyp#L33
Maybe dockcross sets some flag that overrides this? Maybe we need to set additional flags like -fno-unwind-tables
? If it turns out we can't do without _Unwind_Resume
, then this SO suggests we need to statically link libgcc_eh
(which defines _Unwind_Resume
). Or maybe it's an NDK mismatch. We build against ANDROID_NDK_REVISION=16b ANDROID_NDK_API=21
. I have no idea what the implications are.
@ralphtheninja any clue?
@christianbundy Can you send your leveldown.node
compiled from source, or share the output of readelf -a leveldown.node
? We can have a look at its symbol table and compare it against the prebuild. Of our current prebuilds, only android-arm
and android-arm64
reference _Unwind_Resume
:
# readelf -a prebuilds/android-arm64/node.napi.armv8.node | grep _Unwind_Resume
00000006ed90 024c00000402 R_AARCH64_JUMP_SL 0000000000000000 _Unwind_Resume + 0
588: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _Unwind_Resume
# readelf -a prebuilds/android-arm/node.napi.armv7.node | grep _Unwind_Resume
0005e160 00000316 R_ARM_JUMP_SLOT 0004dec9 _Unwind_Resume
3: 0004dec9 136 FUNC GLOBAL DEFAULT 11 _Unwind_Resume
Maybe dockcross sets some flag that overrides this? __
We could start by checking the verbose output during compile time, e.g. npm i --verbose
to see the flags passed on to gcc.
This line seems to indicate that we explicitly set exceptions for Android
The leveldown/binding.gyp
only applies to the c++ code in leveldown
, e.g. binding.cc
.
Oh right, I forgot about that. What's the difference between ccflags
(where we have -fexceptions
) and cflags!
(where we have -fno-exceptions
)?
@#$@#$% knows. We need to do some .gyp
digging :man_dancing:
Output of npx prebuildify-cross -i android-arm64 -t 8.14.0 --napi --strip
with loglevel silly:
Noteworthy:
clang50++: warning: argument unused during compilation: '-pie'
-fno-exceptions
or -fexceptions
flagExtract of one of the compiled leveldb files:
/usr/aarch64-linux-android/bin/aarch64-linux-android-clang++
'-DNODE_GYP_MODULE_NAME=leveldb'
'-DUSING_UV_SHARED=1'
'-DUSING_V8_SHARED=1'
'-DV8_DEPRECATION_WARNINGS=1'
'-D_LARGEFILE_SOURCE'
'-D_FILE_OFFSET_BITS=64'
'-DSNAPPY=1'
'-DLEVELDB_PLATFORM_POSIX=1'
'-DOS_ANDROID=1'
'-D_REENTRANT=1'
'-D_GLIBCXX_USE_C99_MATH'
[.. linker flags ..]
-fPIC -Wall -Wextra -Wno-unused-parameter -std=c++0x
-Wno-sign-compare -fPIC -O3 -fno-omit-frame-pointer
-fno-rtti -std=gnu++0x -MMD -MF
./Release/.deps/Release/obj.target/leveldb/deps/leveldb/leveldb-1.20/db/builder.o.d.raw
-c -o Release/obj.target/leveldb/deps/leveldb/leveldb-1.20/db/builder.o
../deps/leveldb/leveldb-1.20/db/builder.cc
What's the difference between
ccflags
(where we have-fexceptions
) andcflags!
(where we have-fno-exceptions
)?
The "!" means: exclude -fno-exceptions
from cflags. So a correction: on both our binding.cc
and leveldb files, exceptions are enabled.
So maybe then we should take care to link with whatever is needed.
And add -fvisibility=hidden
while we're at it. If nothing else, that'll reduce binary size a little bit.
I'd like to explore the option of disabling exceptions for all platforms. As far as I can tell we don't need them (anymore?).
That is as expected
Sweet. I assumed that it would either use a prebuild or compile from source, but didn't realize there was a smoke test going on.
Snappy is a git submodule.
Love it, thanks. I should've kept scrolling.
Can you send your leveldown.node compiled from source, or share the output of readelf -a leveldown.node?
On it. Thanks for the instructions!
Here's the difference in the symbol tables between your build (A) and the prebuild (B).
In A, the _Unwind_Resume
symbol has an address. Does this suggest the function is inlined? Maybe we can find out with gdb.
In B, the _Unwind_Resume
symbol has no address. The dynamic linker will look for _Unwind_Resume
in a shared library. The error that you get suggests there is no such library.
Relocation section '.rela.plt' (Procedure Linkage Table):
Offset Info Type Sym. Value Sym. Name + Addend
000000063af8 00e700000402 R_AARCH64_JUMP_SL 000000000004ec0c _Unwind_Resume + 0 (A)
00000006ead8 024b00000402 R_AARCH64_JUMP_SL 0000000000000000 _Unwind_Resume + 0 (B)
Symbol table '.dynsym':
Num: Value Size Type Bind Vis Ndx Name
231: 000000000004ec0c 252 FUNC GLOBAL DEFAULT 10 _Unwind_Resume (A)
587: 0000000000000000 0 FUNC GLOBAL DEFAULT UND _Unwind_Resume (B)
@christianbundy For further clues, can you provide the output of $CC -v
and env
?
Maybe at this point we should just test a few different prebuilds. One without exceptions, one with a statically linked libgcc_eh
.
$ cc -v
clang version 9.0.1
Target: aarch64-unknown-linux-android
Thread model: posix
InstalledDir: /data/data/com.termux/files/usr/bin
$ env
SHELL=/data/data/com.termux/files/usr/bin/bash
PREFIX=/data/data/com.termux/files/usr
PWD=/data/data/com.termux/files/home
EXTERNAL_STORAGE=/sdcard
LD_PRELOAD=/data/data/com.termux/files/usr/lib/libtermux-exec.so
HOME=/data/data/com.termux/files/home
LANG=en_US.UTF-8
ANDROID_RUNTIME_ROOT=/apex/com.android.runtime
TMPDIR=/data/data/com.termux/files/usr/tmp
ANDROID_DATA=/data
TERM=xterm-256color
SHLVL=1
ANDROID_ROOT=/system
BOOTCLASSPATH=/apex/com.android.runtime/javalib/core-oj.jar:/apex/com.android.runtime/javalib/core-libart.jar:/apex/com.android.runtime/javalib/okhttp.jar:/apex/com.android.runtime/javalib/bouncycastle.jar:/apex/com.android.runtime/javalib/apache-xml.jar:/system/framework/framework.jar:/system/framework/ext.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/system/framework/android.test.base.jar:/apex/com.android.conscrypt/javalib/conscrypt.jar:/apex/com.android.media/javalib/updatable-media.jar
ANDROID_TZDATA_ROOT=/apex/com.android.tzdata
PATH=/data/data/com.termux/files/home/.npm-global/bin:/data/data/com.termux/files/usr/bin:/data/data/com.termux/files/usr/bin/applets
_=/data/data/com.termux/files/usr/bin/env
I installed termux just now, with nodejs-lts
, leveldown@5.4.1
, on Android 9, ARM 64. Unfortunately I'm unable to reproduce the issue. I tried out gdb
and lldb
too - to see what our debugging options are - but they both crash (unrelated to leveldown
).
Hmm, interesting. Maybe try pkg install nodejs
to see if the problem is happening on Node 13? I was getting "Segmentation fault" on Node 12 and re-upgraded to Node 13. No core dump or anything useful, so the segfault could've been coming from anywhere.
(termux is awesome, btw. first time i tried it)
Same with node 13. So..
Maybe at this point we should just test a few different prebuilds. One without exceptions, one with a statically linked
libgcc_eh
.
@christianbundy would you trust a binary I'd send, or do you want to compile it yourself?
I'd trust a binary, but if it's easier for you I can also compile from source. Makes no difference to me.
A build without exceptions (compiled on prebuild/android-arm64
): node.napi.armv8.tar.gz
To test, run the following commands in an empty directory:
npm i leveldown --ignore-scripts --no-save
curl -OL https://github.com/Level/leveldown/files/4238293/node.napi.armv8.tar.gz
tar -zxvf node.napi.armv8.tar.gz
# Make sure prebuilds are not loaded
rm -rf node_modules/leveldown/prebuilds
mkdir -p node_modules/leveldown/build/Release
mv node.napi.armv8.node node_modules/leveldown/build/Release/leveldown.node
node -e "require('leveldown')('db').open(console.log)"
$ npm i leveldown --ignore-scripts --no-save
npm WARN npm npm does not support Node.js v13.0.0
npm WARN You should probably upgrade to a newer version of node as we
npm WARN can't make any promises that npm will work with this version.
npm WARN Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11, 12.
npm WARN You can find the latest version at https://nodejs.org/
npm WARN enoent ENOENT: no such file or directory, open '/data/data/com.termux/files/home/level-test/package.json'
npm WARN level-test No description
npm WARN level-test No repository field.
npm WARN level-test No README data
npm WARN level-test No license field.
+ leveldown@5.4.1
added 7 packages from 5 contributors in 3.16s
found 0 vulnerabilities
$ curl -OL https://github.com/Level/leveldown/files/4238293/node.napi.armv8.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:-100 147k 100 147k 0 0 82810 0 0:00:01 0:00:01 --:--:-- 216k
$ tar -zxvf node.napi.armv8.tar.gz
node.napi.armv8.node
$
$ # Make sure prebuilds are not loaded
$ rm -rf node_modules/leveldown/prebuilds
$
$ mkdir -p node_modules/leveldown/build/Release
$ mv node.napi.armv8.node node_modules/leveldown/build/Release/leveldown.node
$ node -e "require('leveldown')('db').open(console.log)"
No output. That's good, right? :upside_down_face:
try ls
, there should be a db
$ ls
db node.api.armv8.tar.gz node_modules
!!! You did it. Thanks so much for all the work you've put into this issue.
I'd like to explore the option of disabling exceptions for all platforms. As far as I can tell we don't need them (anymore?).
Aye, I don't see why they should be enabled on Android but not on other platforms.
Thanks again!
Hi! I'm trying to use leveldown on my Android device via Termux, but I'm bumping into this error. Any chance someone knows what this problem might be? I'm using leveldown 5.4.1 on a Google Pixel XL.