irondash / cargokit

Integrate cargo build with flutter plugins and applications.
Other
52 stars 16 forks source link

Flutter, Android, rinf, Cargokit, Rust, C: Problem loading ndk standard C++ library #72

Open ig-garcia opened 3 weeks ago

ig-garcia commented 3 weeks ago

Hello, I am trying to compile for Android a C library that needs to be compiled separately for every Android architecture. This C library is a dependency of a rust library which in turn is also dependency of another rust library which is dependency of the hub crate of rinf. rinf uses Cargokit under the hood. The rust library hub is depending on has a build.rs file where I build the C library.

The C library is building fine, and I can even use it in my crate Rust tests . But when running the Android app, on startup I get this error:

E/flutter (23507): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Invalid argument(s): Failed to load dynamic library 'libhub.so': dlopen failed: cannot locate symbol "_ZNSt6__ndk112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEED1Ev" referenced by "/data/app/~~J4JqwUrdz54b5jQTjobuHg==/security.planck.phoenix-uCymzZHJOoinr6TH7XUveg==/base.apk!/lib/arm64-v8a/libhub.so"...
E/flutter (23507): #0      _open (dart:ffi-patch/ffi_dynamic_library_patch.dart:11:43)
E/flutter (23507): #1      new DynamicLibrary.open (dart:ffi-patch/ffi_dynamic_library_patch.dart:22:12)
E/flutter (23507): #2      loadRustLibrary (package:rinf/src/load_os.dart:25:27)
E/flutter (23507): #3      rustLibrary (package:rinf/src/load_os.dart:5:21)
E/flutter (23507): #4      rustLibrary (package:rinf/src/load_os.dart)
E/flutter (23507): #5      prepareInterfaceReal (package:rinf/src/interface_os.dart:21:24)
E/flutter (23507): #6      initializeRust (package:rinf/rinf.dart:17:9)

The problem seems to be that the build is not linking with the standard ndk C++ library (like libc++_shared.so).

Also in my Rust crate's build.rs I printed the env variables, and some are set by Cargokit. I get some env vars that look a bit strange, like for example:

CC_aarch64-linux-android: /Users/ignaciogarcia/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang

Somehow the target arch is not included in these vars... like I would be expecting something like

/Users/ignaciogarcia/Library/Android/sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/darwin-x86_64/bin/aarch64-linux-android21-clang

Please see the difference: the second path has included aarch64-linux-android21- which are the target triplet + the min android version.

When I tried to use these env vars for my compilation it did not work, so I used the ones that seemed to work, as I provided above, some of them include target arch, etc.

So I have a few questions:

Thanks in advance, regards.

knopp commented 3 weeks ago

The target (and minimum version) is specified in cflags (i.e. --target=aarch64-linux-android21). If you look at the aarch64-linux-android21-clang file it's just a wrapper that adds the --target argument.

knopp commented 3 weeks ago

Also the linker error looks like you need to link with libc++, so a missing linker flag, but Rust doesn't use libc++ so I'd assume that whichever crate is compiling and linking c++ code should take care of that.

ig-garcia commented 3 weeks ago

Also the linker error looks like you need to link with libc++, so a missing linker flag, but Rust doesn't use libc++ so I'd assume that whichever crate is compiling and linking c++ code should take care of that.

The C library I am compiling is not compiling inside Rust itself, it is compiled in the build.rs file of my Rust crate but I am using the Rust std::process::Command crate to actually call the make and make install commands of the C library, passing to it the parameters needed for each Android arch.

knopp commented 3 weeks ago

I think on Android you can either manually link with libc++_static.a, or have to include libc++_shared.so in your APK.

https://developer.android.com/ndk/guides/cpp-support#selecting_a_c_runtime

ig-garcia commented 3 weeks ago

The target (and minimum version) is specified in cflags (i.e. --target=aarch64-linux-android21). If you look at the aarch64-linux-android21-clang file it's just a wrapper that adds the --target argument.

Finally I verified our C library does not like the format of the compiler + flags from Cargokit, so I will set them manually from paths.

I see there is a env var called CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER, is it supposed to be used as the LD (linker) in the environment or similar?

ig-garcia commented 3 weeks ago

I think on Android you can either manually link with libc++_static.a, or have to include libc++_shared.so in your APK.

https://developer.android.com/ndk/guides/cpp-support#selecting_a_c_runtime

I actually included the libc++_shared.so in the APK, in the arm64-v8a folder, but somehow when is packaged into the APK the library loses all its symbols. On the other hand, if I copy same library to an aarch64 folder the symbols are kept, but of course Android does not use it.

knopp commented 3 weeks ago

Finally I verified our C library does not like the format of the compiler + flags from Cargokit, so I will set them manually from paths.

I have no idea what that means. The --target argument is a clang argument, nothing to do with your library, unless there is no way to pass custom compile flags, which woould be very weird.

I actually included the libc++_shared.so in the APK, in the arm64-v8a folder, but somehow when is packaged into the APK the library loses all its symbols.

gradle strips libraries (there is a way to configure it not to at project level), but public symbols should not get stripped.