hyperxpro / Brotli4j

Brotli4j provides Brotli compression and decompression for Java.
Apache License 2.0
106 stars 35 forks source link

Compiling native JNI code for Android? #58

Open lukeaschenbrenner opened 1 year ago

lukeaschenbrenner commented 1 year ago

Hello, I am a relatively rookie developer working on an app for Android that requires the use of brotli encoding/compression. Google has already provided a brotli decoding library that works with Android ( https://mvnrepository.com/artifact/org.brotli/dec/0.1.2 ) but encoding isn't implemented in their library. What steps would one need to take to create native code (something to do with JNI?) that works on Android in order to use Brotli4j? Thank you!

hyperxpro commented 1 year ago

If I'm correct then you can use Brotli4j linux_aarch64 for running on Android. No need for any extra compilations.

lukeaschenbrenner commented 1 year ago

Thank you for the reply, I will test and report back soon.

lukeaschenbrenner commented 1 year ago

@hyperxpro Just a quick update: I tried to run Brotli4jLoader.ensureAvailability() but the code threw this exception: java.lang.UnsatisfiedLinkError: Failed to load Brotli native library at com.aayushatharva.brotli4j.Brotli4jLoader.ensureAvailability(Brotli4jLoader.java:79) I'm guessing this means that it was unable to find the JNI bindings that worked with Android? For reference, this is the relevant part of my current app module's build.gradle file:

    implementation("com.aayushatharva.brotli4j:brotli4j:1.8.0")
    runtimeOnly(
            "com.aayushatharva.brotli4j:native-linux-aarch64:1.8.0"
    )

Please let me know if I did anything wrong! (Just to double check, it is my understanding that the enclosed import in runtimeOnly makes sure only the aarch64 linux bindings are compiled when building an APK, but when running a build in Android Studio, the implementation line copies all the bindings over from all platforms? This is my current understanding, but I'm worried that my understanding may be flawed.

Thank you very much for your help, I appreciate it.

hyperxpro commented 1 year ago

Can you show full stack trace so it'll be easier to debug?

lukeaschenbrenner commented 1 year ago

Sure thing! This was the full stack trace from the error:

2022-11-16 18:19:38.498 6455-6455/com.txtnet.txtnetbrowser D/AndroidRuntime: Shutting down VM
2022-11-16 18:19:38.500 6455-6455/com.txtnet.txtnetbrowser E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.txtnet.txtnetbrowser, PID: 6455
    java.lang.UnsatisfiedLinkError: Failed to load Brotli native library
        at com.aayushatharva.brotli4j.Brotli4jLoader.ensureAvailability(Brotli4jLoader.java:79)
        at com.txtnet.txtnetbrowser.MainBrowserScreen$7.onClick(MainBrowserScreen.java:287)
        at android.view.View.performClick(View.java:7455)
        at android.view.View.performClickInternal(View.java:7432)
        at android.view.View.access$3700(View.java:835)
        at android.view.View$PerformClick.run(View.java:28810)
        at android.os.Handler.handleCallback(Handler.java:938)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loopOnce(Looper.java:201)
        at android.os.Looper.loop(Looper.java:288)
        at android.app.ActivityThread.main(ActivityThread.java:7870)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1003)
     Caused by: java.lang.NullPointerException
        at java.util.Objects.requireNonNull(Objects.java:220)
        at java.nio.file.Files.copy(Files.java:2984)
        at com.aayushatharva.brotli4j.Brotli4jLoader.<clinit>(Brotli4jLoader.java:50)
        at com.aayushatharva.brotli4j.Brotli4jLoader.ensureAvailability(Brotli4jLoader.java:78)
        at com.txtnet.txtnetbrowser.MainBrowserScreen$7.onClick(MainBrowserScreen.java:287) 
        at android.view.View.performClick(View.java:7455) 
        at android.view.View.performClickInternal(View.java:7432) 
        at android.view.View.access$3700(View.java:835) 
        at android.view.View$PerformClick.run(View.java:28810) 
        at android.os.Handler.handleCallback(Handler.java:938) 
        at android.os.Handler.dispatchMessage(Handler.java:99) 
        at android.os.Looper.loopOnce(Looper.java:201) 
        at android.os.Looper.loop(Looper.java:288) 
        at android.app.ActivityThread.main(ActivityThread.java:7870) 
        at java.lang.reflect.Method.invoke(Native Method) 
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548) 
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1003) 
2022-11-16 18:19:38.512 6455-6455/com.txtnet.txtnetbrowser I/Process: Sending signal. PID: 6455 SIG: 9

Let me know if that's not enough, or if you need me to test something.

lukeaschenbrenner commented 1 year ago

Hello, Just checking in to see if you have any updates on how this stack trace should be interpreted.

hyperxpro commented 1 year ago

Sorry I missed this. I'll get back to you with a solution.

lukeaschenbrenner commented 1 year ago

Thank you for responding! No problem. Let me know if I can help in any way; I would be interested in learning more about compiling the binaries you provide for different target architectures. I discovered a port of Brotli to Haxe, which compiles to Java source code. I have yet to test it on Android but it looks to be platform independent. If an Android ARMv7 binary isn't possible with BrotliJ4, at least I will have an alternative.

hyperxpro commented 1 year ago

Sorry to say this I won't be supporting ARMv7 build for a while. But I will definitely consider this in the future.

lukeaschenbrenner commented 1 year ago

I understand, thank you for letting me know and I appreciate that you're looking into it for the future. Do you have any links or resources that might help to explain the process of compiling native code to work with Java using JNI bindings? I don't know if I'd be able to, but I'd like attempt to learn the basics myself and maybe take a shot at compiling for ARMv7 myself.

hyperxpro commented 1 year ago

You can take a look at this PR https://github.com/hyperxpro/Brotli4j/pull/66 to see how it is gonna work.

I am also trying because I don't have first-hand experience in compiling for ArmV7. :(

hyperxpro commented 1 year ago

I added the ARMv7 module. PTAL #67 :)

lukeaschenbrenner commented 1 year ago

Thank you so much! I attempted to build the object file for ARMv7 myself but the build failed. It seems as though the GitHub actions for the JDK17 docker action in the armv7 branch failed too (Docker seems to be running for that build on amd64, so it can't compile for arm32v7). I didn't see any generated artifacts for the JDK 8 build either. It's possible this is a simple fix, but I have never used GitHub actions before.

hyperxpro commented 1 year ago

I'm fixing this. I will ping you when its ready.

lukeaschenbrenner commented 1 year ago

I believe the library compiled correctly, sadly I am unable to load it on Android because the Android C++ linking system requires specific flags and arguments that require the Android NDK to build correctly. (eg. for some reason there is no libstdc++.so, only libc++_shared.so. It seems like I'm going to have to attempt to build the Brotli project myself using the Android NDK. I will keep this issue closed unless I uncover some problem in the implementation here. Thanks for your help getting ARMv7 support into the main project though, I appreciate it!

hyperxpro commented 1 year ago

Which Android version you're trying to run?

lukeaschenbrenner commented 1 year ago

I tested my app as both an ARMv7-only executable and ARMv8 (native) executable on a device running Android 13. It looks like Brotli would need to be compiled differently to run on any version of android at all though. This was the website I referenced: https://developer.android.com/ndk/guides/cpp-support

hyperxpro commented 1 year ago

Can you do a PR?

lukeaschenbrenner commented 1 year ago

I will do a PR once I finish compiling it myself. Right now I'm moving forward with a Java-native port of Brotli instead, but within the next couple months I will tackle compiling Brotli in C in a way that makes it compatible with the Android NDK. I will let you know when/if I finish!

moqmar commented 1 year ago

Is there any update on this? Seems like this is right now the most promising thread on the internet regarding Brotli compression on Android 🙈

hyperxpro commented 1 year ago

Is there any update on this? Seems like this is right now the most promising thread on the internet regarding Brotli compression on Android 🙈

Armv7 doesn't help?

lukeaschenbrenner commented 1 year ago

Is there any update on this? Seems like this is right now the most promising thread on the internet regarding Brotli compression on Android 🙈

I apologize for not sharing earlier, but I was able to port Brotli4J to Android Studio build tools and compile as a .aar library. Works across armv7/v8/x86 Android. You can see the port in use in my project TxtNet Browser, and the source code for the port is available here. Maybe with a little effort I can help get this merged into the build workflow of Brotli4J? I believe you'd need a separate toolchain setup and I haven't looked into automating it yet though.

hyperxpro commented 1 year ago

I had a look at your repo. Indeed we need to work on splitting native code and integrating with a separate build pipeline. I don't have first-class experience with Android but I'll help with infra needed for this.

I have created a branch separately for this, please target a PR for it: https://github.com/hyperxpro/Brotli4j/tree/android

hyperxpro commented 1 year ago

@Ghanshyam32 PTAL