Building native code without rebuilding for every Gradle module/project

[oysteinmyrmo]

We are using cmake to build our C++ code. For our Android builds we have the current dependency setup in cmake for the native code:

1. libNative.so (AAR)
2. libJni.so (links to libNative.so) (AAR)
3. libDemo.so (links to libJni.so) (APK)

Let's say the different Gradle modules for the three targets. Now, if we do this for the Native module, targeting libNative.so:

externalNativeBuild {
    cmake {
        targets "Native"
    }
    ...
}

followed by ./gradlew :Native:assembleRelease we get the proper AAR file with libNative.so inside the jni directory. Next we do the following for the Jni module, targeting libJni.so:

externalNativeBuild {
    cmake {
        targets "Jni"
    }
    ...
}
...
dependencies {
    compile project(":Native")
}

This would build libNative.so and libJni.so (i.e. it rebuilds libNative.so in the build directory for the Jni module). This is not desired as it increases the build time a lot. We would expect this build setup to take the output files (be it AAR files or so files directly) from the Native module and include them in the build for the Jni module and perform the linking accordingly.

Then we could do the following for the Demo target and for any other application regardless of where the code is relative to the the Native and Jni modules:

externalNativeBuild {
    cmake {
        targets "Demo"
    }
    ...
}
...
dependencies {
    compile project(":Jni") // Assuming that the proper path to the Jni module is set in settings.gradle
}

Our current workaround is to build all native code in the Native module, then copy all the so files from that module to somewhere and then have copy tasks in all other Gradle modules and projects. This works, but it creates a lot of mental overhead and unnecessary maintenance.

Is it possible to do this today somehow? If not, is this planned sometime in the future?

[ggfan]

dependency chain is: demo->jni->native.

In this three components, your project has both dependency for Java side AND native side, or the dependency is purely on native code side? If dependency is purely on Java side, the dependency would go with AAR dependency way, no need to worry about the native side. In this case the only thing to worry is to make sure the .so files are packed inside APK, which would be done automatically ( AAR dependency handles that ) If dependency is purely on native side, CMake's domain could handle it nicely ( add_subdirectory() + target_link_libraries()) if dependency is BOTH, your shared lib not only be packed into APK, but also is consumed during the compile time. I would do some experiment first, let me know

[oysteinmyrmo]

Good point. We have multiple possible paths for our code as we both have customers who creates applications with our code, either purely in C++ (with a tiny bit of Java of course), purely in Java or somewhere in between. It is also possible to use Xamarin and other similar frameworks. Extending on our libs described above our base build output is:

1. libNative.so: C++ only, this is the main library we provide. Is provided as a standalone so file and as an AAR file containing libNative.so only.
2. libJni.so: Both C++ and Java, contains the C++ side and Java side code to call into libNative.so from Java and from libNative.so back to the Java side. libJni.so is linked to libNative.so in cmake by doing target_link_libraries(Jni Native). Is provided as an AAR containing both C++ libs and the Java code, call it jni.aar.

With these two targets/libraries in mind, here are some possible configurations from the user's perspective:

A. Creating a pure C++ application using Qt, Juce or something similar. In this case the user would either link directly to libNative.so or include the AAR that contains libNative.so only. B. Creating a pure Java application by using our built-in JNI code by including jni.aar. C. Create an application that has both its own C++ code and its own Java code, calling into libNative.so through Java and/or C++. D. Using something else like Xamarin. In such cases the user could do many things, so it should be safe to assume that this will be the same use case as one of the above cases.

So the dependency is both for C++ and Java, which makes it a bit more complicated.

[ggfan]

begin looking at this one...

[ggfan]

is this one close to what you saw? https://github.com/googlesamples/android-ndk/tree/issue423/issue423 thx for confirm.

Some observations about it:

• only the very first time gradlew:jniLibAssembleDebug would build native's code; following times (2nd, 3rd) will not trigger build of native anymore
• did not observe this behaviour inside Android Studio IDE (right hand side of IDE: gradle -> :jniLib -> build assembleDebug)

this seems to be just a bug. If this bug is not there, then you build system will not have problem ( my assumption ). To request a totally new native code ONLY build channel would be a difficult one, specially if there is a possible solution to it.

Created a bug at studio page: https://b.corp.google.com/issues/65096095, this issue could be followed there, feel free to add more info to it.

[suikki]

@ggfan Sorry to comment on an old closed issue, but I've been trying to search all over the place for hints how to link to native libs in an AAR from an android gradle cmake project. I'm just commenting because the AAR that I have contains both Java classes and native .so libraries and you said you would be looking into that kind of situation, but both the links mentioned are dead (or behind sign-in).

I could also make the library project a dependency, but the problem is that the library is built with ndk-build and the application using it is using cmake so add_subdirectory is out of the question.

[ggfan]

it is fine to re-use an closed issue, no worries.

The tough part is: I did not see a clean solution. The first link was dead, thank you for pointing it out ( removed: I need find it later ). The second one is internal one. May you kindly do me a favor to create a bug from here: https://issuetracker.google.com/issues/new?component=192708&template=840533 more knowledgeable people will be able to see it. you may also kindly link your newly created bug to this issue here.

thank you so much! it is tough one

[suikki]

Thanks for the quick reply. There is already an issue in the issue tracker that I think would probably solve this, but it hasn't been updated lately: https://issuetracker.google.com/issues/37086680

[oysteinmyrmo]

I added a comment on that issue to maybe give it a bit more traction. We will just have to see I suppose.

[ggfan]

Yeah, that is the right place to discuss for features, thanks! Sorry not able to help for this one

[ggfan]

@suikki now for your specific comment about ndk-build vs. cmake. If you could make dependency and rebuild, it is fine. Android Studio/Gradle supports ndk-build same as cmake: https://developer.android.com/studio/projects/gradle-external-native-builds.html assume you could re-built ( or otherwise ) with ndk-build + gradle, does that workaround the problem? A fully supported AAR for native lib + header files + java lib might take some time.

some examples for gradle + ndk-build is in: https://github.com/googlesamples/android-ndk/tree/master-ndkbuild ( it is the same repo with different branch, use 2.3 to build -- I need find time to update it to 3.0 for productFlavor changes )

[suikki]

Yeah, thanks. I've got a working build now, but it's a bit of a hack. Hopefully I can clean it up in the future.