Build error: Bult Android binaries do not load

elizagamedev commented 8 months ago

Description

This problem happens both when building libmozcjni.so via the Docker build instructions and when using one of the github CI build artifacts. The generated libmozcjni.so contains a great deal of undefined function references but no links to any libraries which define those functions. The result is that attempting to load this library on Android results in an error when calling System.loadLibrary().

Commit-id

006084cf4e3a31a37f48a55c7d4a73790ce39dfb (latest)

Build target

Choose one of them 1. Docker build for Linux and Android-lib

https://github.com/google/mozc/blob/master/docs/build_mozc_in_docker.md
1. macOS build
https://github.com/google/mozc/blob/master/docs/build_mozc_in_osx.md
1. Windows build
https://github.com/google/mozc/blob/master/docs/build_mozc_in_windows.md
1. Others (no guarantee)

CI build status

Whether the current CI build status is passing or failure. https://github.com/google/mozc#build-status

Choose either of them

passing
failure

Environment

Docker image built from latest revision

Build commands

Following exactly https://github.com/google/mozc/blob/master/docs/build_mozc_in_docker.md.

Error logs

Here is the dlopen error generated from attempting to call loadLibrary on the latest CI artifact.

java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "_ZNSt6__ndk15ctypeIcE2idE" referenced by "/data/app/~~R5usQJuIgq1kTfPJpnv8FA==/org.mozc.android.inputmethod.japanese-W6Ui47b9v4YKwX352Z3TcA==/base.apk!/lib/arm64-v8a/libmozcjni.so"...

Additional context

Here is the output of objdump -T on the latest CI artifact for Android. (Only the relevant lines are included.)

0000000000000000      DF *UND*  0000000000000000 (LIBC)       __cxa_finalize
0000000000000000      DF *UND*  0000000000000000 (LIBC)       __cxa_atexit
0000000000000000      DF *UND*  0000000000000000 (LIBC)       __register_atfork
0000000000000000      DF *UND*  0000000000000000 (LIBC)       memcpy
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc7Logging13InitLogStreamERKNSt6__ndk112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE
0000000000000000      D  *UND*  0000000000000000  Base        _ZNSt6__ndk112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEED1Ev
0000000000000000      D  *UND*  0000000000000000  Base        _ZN6google8protobuf11MessageLite14ParseFromArrayEPKvi
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc12LogFinalizerC1ENS_11LogSeverityE
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc7Logging19GetWorkingLogStreamEv
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc7Logging19GetLogMessageHeaderEv
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc7Logging27GetBeginColorEscapeSequenceENS_11LogSeverityE
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc7Logging18GetLogSeverityNameENS_11LogSeverityE
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc7Logging25GetEndColorEscapeSequenceEv
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc12LogFinalizeranERNS_16WorkingLogStreamE
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc12LogFinalizerD1Ev
0000000000000000      D  *UND*  0000000000000000  Base        _ZNK4mozc8commands7Command12ByteSizeLongEv
0000000000000000      D  *UND*  0000000000000000  Base        _ZNK6google8protobuf11MessageLite16SerializeToArrayEPvi
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc8commands7CommandD1Ev
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc10SystemUtil23GetUserProfileDirectoryEv
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc10SystemUtil23SetUserProfileDirectoryERKNSt6__ndk112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE
0000000000000000      D  *UND*  0000000000000000  Base        _ZNSt6__ndk112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6__initEPKcm
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc8commands7CommandC2EPN6google8protobuf5ArenaE
0000000000000000      D  *UND*  0000000000000000  Base        _ZNSt6__ndk113basic_ostreamIcNS_11char_traitsIcEEElsEi
0000000000000000      D  *UND*  0000000000000000  Base        _ZNSt6__ndk113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_
0000000000000000      D  *UND*  0000000000000000  Base        _ZNSt6__ndk113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev
0000000000000000      D  *UND*  0000000000000000  Base        __cxa_begin_catch
0000000000000000      D  *UND*  0000000000000000  Base        _ZNSt6__ndk18ios_base33__set_badbit_and_consider_rethrowEv
0000000000000000      D  *UND*  0000000000000000  Base        __cxa_end_catch
0000000000000000      D  *UND*  0000000000000000  Base        _ZSt9terminatev
0000000000000000      D  *UND*  0000000000000000  Base        _ZNSt6__ndk112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6__initEmc
0000000000000000      D  *UND*  0000000000000000  Base        _ZNKSt6__ndk18ios_base6getlocEv
0000000000000000      D  *UND*  0000000000000000  Base        _ZNSt6__ndk16localeD1Ev
0000000000000000      D  *UND*  0000000000000000  Base        _ZNSt6__ndk15ctypeIcE2idE
0000000000000000      D  *UND*  0000000000000000  Base        _ZNKSt6__ndk16locale9use_facetERNS0_2idE
0000000000000000      D  *UND*  0000000000000000  Base        _ZNSt6__ndk18ios_base5clearEj
0000000000000000      DF *UND*  0000000000000000 (LIBC)       strlen
0000000000000000      D  *UND*  0000000000000000  Base        _Znwm
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc13MinimalEngineC1Ev
0000000000000000      D  *UND*  0000000000000000  Base        _ZdlPv
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc11DataManager14CreateFromFileERKNSt6__ndk112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc6Engine18CreateMobileEngineENSt6__ndk110unique_ptrIKNS_20DataManagerInterfaceENS1_14default_deleteIS4_EEEE
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc14SessionHandlerC1ENSt6__ndk110unique_ptrINS_15EngineInterfaceENS1_14default_deleteIS3_EEEENS2_INS_22EngineBuilderInterfaceENS4_IS7_EEEE
0000000000000000      DF *UND*  0000000000000000 (LIBC)       memset
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4absl12lts_20230125lsERNSt6__ndk113basic_ostreamIcNS1_11char_traitsIcEEEERKNS0_6StatusE
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4absl12lts_2023012517internal_statusor6Helper5CrashERKNS0_6StatusE
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4absl12lts_202301256Status15UnrefNonInlinedEm
0000000000000000      D  *UND*  0000000000000000  Base        _ZTVN4mozc13EngineBuilderE
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc18SingletonFinalizer12AddFinalizerEPFvvE
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4mozc7session20SessionUsageObserverC1Ev
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4absl12lts_2023012516raw_log_internal6RawLogENS0_11LogSeverityEPKciS4_z
0000000000000000      D  *UND*  0000000000000000  Base        _ZN4absl12lts_2023012513base_internal12SpinLockWaitEPNSt6__ndk16atomicIjEEiPKNS1_22SpinLockWaitTransitionENS1_14SchedulingModeE
0000000000000000      D  *UND*  0000000000000000  Base        AbslInternalSpinLockWake_lts_20230125
0000000000000000      D  *UND*  0000000000000000  Base        _ZNSt6__ndk112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE9__grow_byEmmmmmm
0000000000000000      D  *UND*  0000000000000000  Base        __cxa_pure_virtual
0000000000000000      D  *UND*  0000000000000000  Base        _ZTVN10__cxxabiv117__class_type_infoE
0000000000000000      D  *UND*  0000000000000000  Base        __gxx_personality_v0
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_rwlock_unlock
0000000000000000      DF *UND*  0000000000000000 (LIBC)       fprintf
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_rwlock_wrlock
0000000000000000      DF *UND*  0000000000000000 (LIBC)       strncmp
0000000000000000      DF *UND*  0000000000000000 (LIBC)       malloc
0000000000000000      DF *UND*  0000000000000000 (LIBC)       __system_property_get
0000000000000000      DF *UND*  0000000000000000 (LIBC)       free
0000000000000000      DF *UND*  0000000000000000 (LIBC)       fflush
0000000000000000      DF *UND*  0000000000000000 (LIBC)       getauxval
0000000000000000      DF *UND*  0000000000000000 (LIBC)       pthread_rwlock_rdlock
0000000000000000      DF *UND*  0000000000000000 (LIBC)       abort
0000000000000000      DF *UND*  0000000000000000 (LIBC)       fwrite
0000000000000000      DO *UND*  0000000000000000 (LIBC)       __sF
0000000000000000      DF *UND*  0000000000000000 (LIBC)       dl_iterate_phdr

A good number of these appear to be libc++ functions. However, libmozcjni.so doesn't link to libc++ according to readelf -d.

Dynamic section at offset 0x17150 contains 26 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libc.so]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so]
 0x000000000000001e (FLAGS)              BIND_NOW
 0x000000006ffffffb (FLAGS_1)            Flags: NOW
 0x0000000000000007 (RELA)               0x5368
 0x0000000000000008 (RELASZ)             1248 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffff9 (RELACOUNT)          32
 0x0000000000000017 (JMPREL)             0x5848
 0x0000000000000002 (PLTRELSZ)           4440 (bytes)
 0x0000000000000003 (PLTGOT)             0x17338
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000006 (SYMTAB)             0x2f0
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000005 (STRTAB)             0x1b74
 0x000000000000000a (STRSZ)              14324 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x17d0
 0x0000000000000019 (INIT_ARRAY)         0x17140
 0x000000000000001b (INIT_ARRAYSZ)       16 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x17130
 0x000000000000001c (FINI_ARRAYSZ)       16 (bytes)
 0x000000006ffffff0 (VERSYM)             0x15f8
 0x000000006ffffffe (VERNEED)            0x1790
 0x000000006fffffff (VERNEEDNUM)         2
 0x0000000000000000 (NULL)               0x0

I may be making a mistake and misunderstanding how elf/Android works, but something here seems wrong to me.

yukawa commented 8 months ago

Thank you for the report.

I think your analysis is right. Seems that trying to build only *.so files with --crosstool_top= and --cpu= option, which I learned from the following section, isn't working as we hoped.

Building a cc_library for Android without using android_binary

Today I played with the other approach, which is also explained in another section in the same page, to follow the way people build an APK with native libraries with Bazel in an end to end manner.

How it works: introducing Android configuration transitions

This approach, which builds an APK with --android_crosstool_top= and --fat_apk_cpu= option instead of --crosstool_top= and --cpu=, indeed worked as expected. The *.so files extracted from the final APK looked OK.

So my plan is to rewrite the current build rule with the above one. It still takes time for me to finish cleaning up my prototype before submitting it, but I'm mostly sure that this is at least a fixable problem.

I'll update this thread when the fix becomes ready.

elizagamedev commented 8 months ago

Thank you so much for the detailed and quick response! I'm not well versed in the Android NDK world, especially when building with bazel, so it would have taken me ages to figure this out. I really appreciate it.

yukawa commented 8 months ago

With b0a604f110e01d11107ebbaad09e674cecee34f5, loading libmozc.so as follows started succeeding in my local environment.

System.loadLibrary("mozc");

Can you confirm that System.loadLibrary("mozc") succeeds without any exception in your environment?

Here is a copy of native_libs.zip downloaded from https://github.com/google/mozc/actions/runs/6727110559

elizagamedev commented 8 months ago

It loads perfectly now. Thank you so much!

google / mozc