search

luben / zstd-jni

JNI binding for Zstd
Other
809 stars 165 forks source link

No zstd-jni-1.5.2-3 in java.library.path (Android tests) #220

Closed carlos-mg89 closed 2 years ago

carlos-mg89 commented 2 years ago

Hi!

First of all, I'd like to state that this library is working perfectly fine in my app. I'm able to decompress ZSTD compressed files in a very short time with no issues registered until the moment (it's been already about 2 years since I changed from ZIP compressed files).

However, today I am trying to create some tests for the class I wrote that decompresses InputStreams. And the tests are failing.

From what I understand by reading the error log, the library cannot be found.

I've tried installing it in my Ubuntu (with apt-get install zstd, and the output of zstd --version is zstd command line interface 64-bits v1.4.8, by Yann Collet. This clearly doesn't match my Android app's library version, for zstd-jni which is 1.5.2-3.

I guess that Gradle gets the libraries to pack them in the Android app, but it doesn't make them available for the Android test runner.

Does anyone know how to deal with it?

This is the complete error log after running the test on Android Studio:

no zstd-jni-1.5.2-3 in java.library.path: [/usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib]
Unsupported OS/arch, cannot find /linux/amd64/libzstd-jni-1.5.2-3.so or load zstd-jni-1.5.2-3 from system libraries. Please try building from source the jar or providing libzstd-jni-1.5.2-3 in your system.
java.lang.UnsatisfiedLinkError: no zstd-jni-1.5.2-3 in java.library.path: [/usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib]
Unsupported OS/arch, cannot find /linux/amd64/libzstd-jni-1.5.2-3.so or load zstd-jni-1.5.2-3 from system libraries. Please try building from source the jar or providing libzstd-jni-1.5.2-3 in your system.
    at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2670)
    at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:830)
    at java.base/java.lang.System.loadLibrary(System.java:1873)
    at com.github.luben.zstd.util.Native.load(Native.java:87)
    at com.github.luben.zstd.util.Native.load(Native.java:55)
    at com.github.luben.zstd.ZstdInputStreamNoFinalizer.<clinit>(ZstdInputStreamNoFinalizer.java:23)
    at com.github.luben.zstd.ZstdInputStream.<init>(ZstdInputStream.java:24)
    at org.apache.commons.compress.compressors.zstandard.ZstdCompressorInputStream.<init>(ZstdCompressorInputStream.java:44)
    at com.myapp.domain.utils.ZstdDecompressorTest.whenUnzippingZstdFileThenNothingHappens(ZstdDecompressorTest.kt:38)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
    at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
    at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
    at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
    at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
    at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
    at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
    at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
    at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
    at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
    at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
    at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
luben commented 2 years ago

I don't know how to deal with it. Here are few remarks that may help you:

I did small Android example https://github.com/luben/ZstdAndroidExample but I have not tried it recently and there is also no tests for it.

One option you can try. Get the jar manually from Maven https://repo1.maven.org/maven2/com/github/luben/zstd-jni/1.5.2-3/zstd-jni-1.5.2-3-linux_amd64.jar , unpack it tell Zstd to use this exactly binary, you have to add -DZstdNativePath=/the/path/to/the/libzstd-jni-1.5.2-3.so to your java options when running the tests

carlos-mg89 commented 2 years ago

Well, partly thanks to your help I've been able to deal with the situation.

There was indeed some issues with the zstd-jni library been loaded. However, the solution was simpler than I expected, although I'm unsure about why is that.

When using zstd-jni for your Android app, getting the library at the Gradle file is done as described in the README:

implementation("com.github.luben:zstd-jni:1.5.2-3@aar")

However, for it to work (so to be loaded) while running tests, this should be added as well:

testImplementation("com.github.luben:zstd-jni:1.5.2-3")

Basically, there's some issues that make the library to not be available while loading the AAR.

Perhaps you would like to add this in the documentation.

luben commented 2 years ago

Aaa, good finding - I will add it to the Android example