Can't compile for Android since recent version of MCL (> 1.60)

[ndesmoul]

I cannot compile MCL for Android after version 1.60 using CMake. I was using a CMakeList.txt inspired from the one provided in https://github.com/herumi/mcl-android. It was working fine with MCL 1.60. But I recently tried to compile with a more recent version and got a lot of linking errors:

../mcl/include/mcl/bint_proto.hpp:(.text._ZN3mcl4bint7get_subEm[_ZN3mcl4bint7get_subEm]+0x18): undefined reference to `mclb_sub1'
../mcl/include/mcl/bint_proto.hpp:(.text._ZN3mcl4bint7get_subEm[_ZN3mcl4bint7get_subEm]+0x1c): undefined reference to `mclb_sub1'
../mcl/include/mcl/bint_proto.hpp:(.text._ZN3mcl4bint7get_subEm[_ZN3mcl4bint7get_subEm]+0x38): undefined reference to `mclb_sub2'
../mcl/include/mcl/bint_proto.hpp:(.text._ZN3mcl4bint7get_subEm[_ZN3mcl4bint7get_subEm]+0x3c): undefined reference to `mclb_sub2'
../mcl/include/mcl/bint_proto.hpp:(.text._ZN3mcl4bint7get_subEm[_ZN3mcl4bint7get_subEm]+0x58): undefined reference to `mclb_sub3'
../mcl/include/mcl/bint_proto.hpp:(.text._ZN3mcl4bint7get_subEm[_ZN3mcl4bint7get_subEm]+0x5c): undefined reference to `mclb_sub3'
../mcl/include/mcl/bint_proto.hpp:(.text._ZN3mcl4bint7get_subEm[_ZN3mcl4bint7get_subEm]+0x78): undefined reference to `mclb_sub4'
[...] 

Could you provide up to date instructions to compile MCL on Android and more generally on ARM(64)?

[herumi]

I'm sorry that I can't fix it now because I'm busy. I think that you add bint64.ll to CMakfile.txt like base64.ll. Or, https://github.com/herumi/bls/tree/master/android/jni contains all mcl functions. Could you try it?

[ndesmoul]

I tried without success. But I'm not even sure that CMake tries to compile the file bint64.ll...

Actually I have serious doubt about the CMakeList.txt provided https://github.com/herumi/mcl-android: if I use mcl 1.60 and comment any reference to base64.ll, it still compiles. I didn't try this version on any smartphone though.

Doing the same thing with last version of MCL doesn't work though, with the same errors than above.

With base64.ll enabled, I can't find any base64.o in intermediate compiled objects... Furthermore introducing errors in either base64.ll or bint64.ll doesn't change anything: no compilation errors, only linking errors. (And it still compiled with mcl 1.60...)

So my assumption is that the .ll files are just ignored during compilation with CMake.

I'm not familiar with .ll files. What tool is supposed to compile or whatever those files? I don't know if there is something missing or wrong with my Android toolchain definitions (it uses Clang), or if there is some additional parameters needed in CMakeList.txt.

[herumi]

It took me a while to familiarize myself with the Android environment. https://github.com/herumi/mcl-android/ can support the latest mcl. At first, type make android in the mcl directory to build static libraries and build the sample program.

[ndesmoul]

No success for now.

make android return an error: make: *** [Makefile:452: android] Error 2

android/jni is not found. If I update line 452 in Makefile like this: @$(NDK_BUILD) -C ffi/java/android/jni NDK_DEBUG=0 MCL_LIB_SHARED=$(MCL_LIB_SHARED)

It goes further, but I have similar undefined references than before.

[herumi]

I'm sorry to miss updating mcl. Could you update mcl-android and

git submodule update --init --recursive

and try it again?

[ndesmoul]

It seems to compile this time. But I think that the the CPP flags "-DCYBOZU_DONT_USE_EXCEPTION -DCYBOZU_DONT_USE_STRING" will be incompatible with my own code.

I tried again to compile using CMake and made some progress. The problem was that I needed to include the base64.o and bint64.o in the addlibrary() (and not the .ll).

set(CXX $ENV{CXX})

add_custom_command(OUTPUT base64.o
        COMMAND ${CXX} -c -o ${CMAKE_CURRENT_BINARY_DIR}/base64.o ../${MCL_HOME}/src/base64.ll -O3
        #DEPENDS  base64.ll 
        WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})

add_custom_command(OUTPUT bint64.o
        COMMAND ${CXX} -c -o ${CMAKE_CURRENT_BINARY_DIR}/bint64.o ../${MCL_HOME}/src/bint64.ll -O3
        # DEPENDS bint64.ll 
        WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})   

add_library(${LIBRAY_NAME} STATIC 
            ${CMAKE_CURRENT_BINARY_DIR}/base64.o
            ${CMAKE_CURRENT_BINARY_DIR}/bint64.o
            ${MCL_HOME}/src/bn_c384_256.cpp
            ${MCL_HOME}/src/fp.cpp
            ) 

This way a static mcl library is generated, and I have no more the undefined references errors.

But when I try to compile my own C++ code using this library, the linking still fail with some strange errors, for each of my source files:

.text.startup referenced in section .text.startup of CMakeFiles/ppass.dir/home/need7331/devel/pa-crypto/src/attributes.cpp.o: defined in discarded section .text.startup[_ZN3mcl3FpTINS_2bn5local5FpTagELm384EE3op_E] of CMakeFiles/ppass.dir/home/need7331/devel/pa-crypto/src/attributes.cpp.o [...]

I have no idea of what could be the problem.

The same compilation configuration (without bint64) still works with MCL version 1.60.

[herumi]

I've appended how to use C++ APIs in hello-libs.cpp with the same mclbn384_256.a , which contains C++ functions. https://github.com/herumi/mcl-android/commit/d24ae0069c4c0d897a129ff8c7f73b354c67e84c You can use std::exception and std::string in the code.

[ndesmoul]

Your Android project compiles.

As a desperate measure I have directly added my own source C++ code into your Android project. Alas in that case the linking fails:

android2/mcl-android/app/src/main/cpp/src/attributes.cpp:0: warning: relocation refers to discarded section
android2/mcl-android/app/src/main/cpp/src/attributes.cpp:0: warning: relocation refers to discarded section
android2/mcl-android/app/src/main/cpp/src/bb_signature.cpp:0: warning: relocation refers to discarded section
android2/mcl-android/app/src/main/cpp/src/bb_signature.cpp:0: warning: relocation refers to discarded section
...
/home/need7331/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: treating warnings as errors
  clang++: error: linker command failed with exit code 1 (use -v to see invocation)
  ninja: build stopped: subcommand failed.

Which I think is similar than the "discarded section" problem I have when I try to compile my code with CMake (using NDK toolchain).

I have no idea why it works with mcl 1.60, and not with more recent versions on Android.

[herumi]

I tried mcl-android with two cpp files, then we had to disable priority attribute in fp.hpp. Then, it runs well. Could you try it?

[ndesmoul]

Thanks. I applied the change in fp.hpp. Now it indeed compiles!

For "armeabi-v7a" architecture I had an undefined method '__strlen_chk'. I had to add another redefinition in hello-lib.cpp:

extern "C" size_t __strlen_chk(const char* const s, size_t dstlen) { return strlen(s); }

For "arm64-v8a" architecture those redefinitions are useless.

I didn't had time to test if the compiled library was correctly working though.

[ndesmoul]

I tried my own application with a compilation of the library using only CMake, and with the change in fp.hpp, it compiles and run correctly, at least for arm64-v8a architecture.

[herumi]

__strlen_chk is not necessary if mcl is built with _FORTIFY_SOURCE=0.