search

RGB-WG / rgb-sdk

RGB SDK for wallet & exchange integration
MIT License
29 stars 18 forks source link

Fix Android integration #5

Closed zoedberg closed 3 years ago

zoedberg commented 3 years ago

After updating android integration in this project, trying run ./gradlew build I've encountered this error:

   Compiling lnpbp v0.1.0-rc.1 (https://github.com/LNP-BP/rust-lnpbp?branch=master#7b7284a4)
error: /mnt/dmc/zoe/work/LNP-BP/rgb-sdk/rust-lib/target/debug/deps/libamplify_derive-3530b0645a331e12.so: undefined symbol: __errno
  --> /home/zoe/.cargo/git/checkouts/rust-lnpbp-1c6b6ac6d4fb658b/7b7284a/src/lib.rs:36:1
   |
36 | extern crate amplify_derive;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

error: could not compile `lnpbp`

After a cargo clean the same error can appear for lnpbp_derive. Both dependencies are proc-macro and are lnp-bp forks. The symbol __errno is defined in libc.so but in libamplify_derive_xxx.so and liblnpbp_derive_xxx.so the symbol is actually undefined and unlinked (inspected with nm). Same error happens when building from rgb-node v0.1.0-rc.1.

To avoid mixing possible project update errors (since integrations have been moved to separate project, there's the possibility I forgot/mistaken to update some paths/commands), I re-added android integration on rgb-node master (as it was before removing it, see this branch) and I receive a different error:

    Updating git repository `https://github.com/LNP-BP/rust-lnpbp`
error: failed to select a version for `secp256k1-sys`.
    ... required by package `secp256k1 v0.18.0`
    ... which is depended on by `bitcoin v0.24.0`
    ... which is depended on by `miniscript v2.0.0`
    ... which is depended on by `lnpbp v0.1.0-rc.2 (https://github.com/LNP-BP/rust-lnpbp?branch=master#82e56fb0)`
    ... which is depended on by `rgb_node v0.1.0-rc.2 (/mnt/dmc/zoe/work/LNP-BP/rgb-node)`
    ... which is depended on by `rgb v0.1.0-rc.1 (/mnt/dmc/zoe/work/LNP-BP/rgb-node/ffi)`
versions that meet the requirements `^0.2.0` are: 0.2.0

the package `secp256k1-sys` links to the native library `rustsecp256k1_v0_2_0`, but it conflicts with a previous package which links to `rustsecp256k1_v0_2_0` as well:
package `secp256k1-sys v0.3.0`
    ... which is depended on by `secp256k1 v0.19.0`
    ... which is depended on by `bitcoin v0.25.0`
    ... which is depended on by `electrum-client v0.3.0-beta.1`
    ... which is depended on by `rgb_node v0.1.0-rc.2 (/mnt/dmc/zoe/work/LNP-BP/rgb-node)`
    ... which is depended on by `rgb v0.1.0-rc.1 (/mnt/dmc/zoe/work/LNP-BP/rgb-node/ffi)`

failed to select a version for `secp256k1-sys` which could resolve this conflict

Which is pretty strange since miniscript v2.0.0 (commit ba436de) has defined bitcoin v0.25.0 (tag lnpbp-v0.1.0-rc1) in its Cargo.toml, not v0.24.0 as the error message reports.

I've tried to cross-compile rust-lnpbp, since it seems to be the package that's causing these problems, and it returns the first error I've reported (undefined symbol: __errno on (lnpbp|amplify)_derive).

On rgb-sdk, in ffi/android/build_rust.sh, I've added a cargo build (without --target, so amd64) on top of the file. That causes the android build to succeed. But then, when loading the library-debug.aar on the android react native demo I receive this error:

E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.react_native_demo, PID: 30090
    java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "_ZNSt6__ndk15ctypeIcE2idE" referenced by "/data/app/com.react_native_demo-xe4-dVhYDMVR1llkVN4klg==/lib/x86/librgb.so"...
        at java.lang.Runtime.loadLibrary0(Runtime.java:1016)
        at java.lang.System.loadLibrary(System.java:1669)
        at com.react_native_demo.MainApplication.onCreate(MainApplication.java:57)
        at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1154)
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5871)
        at android.app.ActivityThread.access$1100(ActivityThread.java:199)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1650)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:193)
        at android.app.ActivityThread.main(ActivityThread.java:6669)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)

That's strange too, since that symbol should be in libc++_shared.so. In order to verify this, I've done the following:

$ANDROID_SDK_ROOT/platform-tools/adb pull "/data/app/com.react_native_demo-xe4-dVhYDMVR1llkVN4klg==/lib/x86/librgb.so" "/home/zoe/work/LNP-BP/librgb_node.so"
$ANDROID_SDK_ROOT/platform-tools/adb pull "/data/app/com.react_native_demo-xe4-dVhYDMVR1llkVN4klg==/lib/x86/libc++_shared.so" "/home/zoe/work/LNP-BP/libc++_shared.so"

$NDK_HOME/toolchains/x86-4.9/prebuilt/linux-x86_64/bin/i686-linux-android-objdump -T ~/work/LNP-BP/librgb.so | less

# piece of output:
00000000      D  *UND*  00000000              _ZNSt6__ndk15ctypeIcE2idE
# it's actually undefined, and unfortunately unlinked

$NDK_HOME/toolchains/x86-4.9/prebuilt/linux-x86_64/bin/i686-linux-android-objdump -T ~/work/LNP-BP/libc++_shared.so | less

# piece of output:
000e6fb0 g    DO .bss   00000008  Base        _ZNSt6__ndk15ctypeIcE2idE
# it's defined

I've added a Dockerfile that may help debugging the situation, both in rgb-sdk and in rgb-node. On rgb-node the behavior is consistent with the local one (fail caused by secp256k1-sys). On rgb-sdk the behavior is inconsistent with the local one, the build is still successful, but the package doesn't contain the jni directory with the *.so files:

unzip -l library-debug.aar
Archive:  library-debug.aar
  Length      Date    Time    Name
---------  ---------- -----   ----
    88185  1980-02-01 00:00   R.txt
      303  1980-02-01 00:00   AndroidManifest.xml
    12663  1980-02-01 00:00   classes.jar
---------                     -------
   101151                     3 files

This actually happened to me also locally, but I haven't determined what causes and fixes that yet.

zoedberg commented 3 years ago

I don't know the reason of many of the reported problems encountered while debugging.

But I've figured out why the library produced from rgb-sdk (with the cargo build on top of build_rust.sh) wasn't linking correctly the libraries. I was accidentally overriding the librgb.so generated from clang++ with the one generated by cargo, by forgetting to remove a cp from build_rust.sh (was necessary only when the project was inside rgb-node).

Now the library gets successfully loaded by the android app. :)

P.S. I'll open a dedicated PR for the mentioned Dockerfile, it still needs some fixes

dr-orlovsky commented 3 years ago

Zoe, the work you've done in investigating and describing thos issues is really important. I am quite sure that many may run into them, and having them sorted out once and for all is really needed. So thank you very much for your efforts!

I am glad that you've managed to fix builder code to get android app linked properly. Just a few comments on the parts of issues above that I've encountered myself and know how to fix, for the history record.

  1. secp256k1 version problem is unrelated to everything else and is caused by downstream dependency bug, which will be solved with the next release. More details can be found here: https://github.com/rust-bitcoin/rust-miniscript/pull/152#issuecomment-706180804 (+ comments above) and the fix tracking issue is here: https://github.com/rust-bitcoin/rust-secp256k1/pull/237#issuecomment-706180042

  2. Before we will do a first full release of both LNP/BP Core Library and RGB Node crates to crates.io we have to rely on fixes for upstream dependencies appended in Cago.toml patch section. This have to be done in every crate using either LNP/BP Core or RGB Node, including SDK. Without it you will get a plenty of cargo complains and error reports. Samples and more details: