search

terl / lazysodium-java

A Java implementation of the Libsodium crypto library. For the lazy dev.
https://github.com/terl/lazysodium-java/wiki
Mozilla Public License 2.0
135 stars 48 forks source link

Running v 4.10 lib search path wrong (?) #56

Closed yishais closed 4 years ago

yishais commented 5 years ago

Might be my misunderstanding... Upgraded from 4.01 to 4.10.

With 4.0.1 the jar will find and load the libsodion.so.

With 4.1.0 is does not. It looks like it looks for it in /lazysodium-java-4.1.0.jar/linux-x86-64/..., but there are only /lazysodium-java-4.1.0.jar/linux64 & /lazysodium-java-4.1.0.jar/linux.

"libsodium.so: cannot open shared object file: No such file or directory Native library (linux-x86-64/libsodium.so) not found in resource path"

4.0,1 & 4.1.0 tried on the same server with the exact same setting, so i suspect it is some setting in the jar.

-Yishai

MakarovS commented 5 years ago

We have the same issue - Native library (linux-x86-64/libsodium.so) not found in resource path. I believe it is because of new library loading introduced in 4.1.0. @gurpreet- could you please take a look at that?

gurpreet- commented 5 years ago

Hi both, can you please use version 4.2.0 as version 4.1.0 had a missing dependency. Please see #55

MakarovS commented 5 years ago

@gurpreet- this issue applies to 4.2.0 as well (I'm using 4.2.0).

gurpreet- commented 5 years ago

How strange! Do you have a full stacktrace that I can see?

I added a new library to manage the handling of resource loading because resource loading is a surprisingly difficult task to do.

MakarovS commented 5 years ago

Sure!

java.lang.ExceptionInInitializerError
        at com.exonum.binding.common.message.TransactionMessageBuilderTest.<clinit>(TransactionMessageBuilderTest.java:47)
Caused by: co.libly.resourceloader.ResourceLoaderException: Failed to load the library using sodium
        at com.exonum.binding.common.message.TransactionMessageBuilderTest.<clinit>(TransactionMessageBuilderTest.java:47)
Caused by: java.lang.UnsatisfiedLinkError:
Unable to load library 'sodium':
libsodium.so: cannot open shared object file: No such file or directory
libsodium.so: cannot open shared object file: No such file or directory
Native library (linux-x86-64/libsodium.so) not found in resource path ([file:/home/.../target/surefire/surefirebooter5421832691743038575.jar])

As OP mentioned - it feels like library loader looks for the library in linux/linux64 folders, while it is in linux-x86-64 folder.

gurpreet- commented 5 years ago

How are you instantiating Lazysodium? What are the lines of code you are using?

LazySodiumJava ls = new LazySodiumJava(new SodiumJava());?

MakarovS commented 5 years ago

Yes, lazySodium = new LazySodiumJava(new SodiumJava(Mode.PREFER_SYSTEM));

gurpreet- commented 5 years ago

Seems to not affect anyone using bundled mode Mode.BUNDLED_ONLY

gurpreet- commented 5 years ago

If you're using it in Mode.PREFER_SYSTEM have you installed libsodium using your package manager? E.g sudo apt install libsodium-dev? Make sure to install Libsodium 1.0.18 or above.

Or you can provide an absolute path to your libsodium.so file: new SodiumJava("/path/to/libsodium.so")

yishais commented 5 years ago

I used new SodiumJava("/path/to/libsodium.so") as a workaround.

It might be that Mode.PREFER_SYSTEM does Mode.SYSTEM_ONLY?

gurpreet- commented 5 years ago

Thank you @yishais so absolute paths worked but not loading from the system. How strange 🤔

Makarov, could you try System.loadLibrary("sodium") before you instantiate lazysodium?

System.loadLibrary("sodium")
LazySodiumJava ls = new LazySodiumJava(new SodiumJava(Mode.PREFER_SYSTEM));

Maybe that might work 🤞

MakarovS commented 5 years ago

I believe Mode.PREFER_SYSTEM doesn't switch to bundled version because of a new library loading introduced in 4.1.0. @gurpreet- please take a look https://github.com/terl/lazysodium-java/pull/57

gurpreet- commented 5 years ago

Hi both,

Version 4.2.1 should fallback correctly now. Sorry for the inconvenience, and thank you for the PR it gave me a lot of inspiration.

MakarovS commented 5 years ago

Will test the new version asap, but the change looks good! Thanks.

MakarovS commented 5 years ago

Unfortunately it still doesn't work correctly :( Here's another PR - https://github.com/terl/lazysodium-java/pull/58

gurpreet- commented 5 years ago

Thank you again @MakarovS!

This is a very annoying bug, I must admit. When I first created this library, I thought the hardest part would be programming it to make it easier for developers to use, but in fact, the hardest part is loading it out of the resources folder as Java and the JAR system make it so difficult! 🤕 I could have leveraged other libraries to load it for me but they were out of date and difficult to use with shared libraries (.so, .dylib and .dll files).

I will create a new release ASAP.

MakarovS commented 5 years ago

@gurpreet- Thank you for maintaining this library and being so responsive :)

gurpreet- commented 5 years ago

@MakarovS,

No problem, it has always been my wish that people use my work 🙂

Version 4.2.2 has your changes in. Thank you for your contributions.

chrisjenx commented 4 years ago

I'm still seeing this on our Jenkins instance, works fine locally on OSx

carbotaniuman commented 4 years ago

Still getting this Native library (linux-x86-64/libsodium.so) not found in resource path

carbotaniuman commented 4 years ago

Appears to just be misleading logging, the actual error is happening somewhere else during loading

gurpreet- commented 4 years ago

Do you have a full stacktrace @carbotaniuman? I'm having difficulty debugging this as it's very OS dependent. Time to fire up my VMs 🙂

carbotaniuman commented 4 years ago

One case is ResourceLoader#getFileFromFileSystem fails with spaces in the name. I'll get you more as I get them.

carbotaniuman commented 4 years ago 
PS F:\BLAH> java -Dfile-level=trace -Dstdout-level=info -jar test.jar
Exception in thread "main" java.lang.UnsatisfiedLinkError: Unable to load library 'C:\Users\Arnold\AppData\Local\Temp\resource-loader15060737488612266253\windows64\libsodium.dll':
The specified module could not be found.

The specified module could not be found.

Native library (win32-x86-64/C:\Users\Arnold\AppData\Local\Temp\resource-loader15060737488612266253\windows64\libsodium.dll) not found in resource path (test.jar)
        at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:302)
        at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:455)
        at com.sun.jna.Native.register(Native.java:1722)
        at co.libly.resourceloader.SharedLibraryLoader.registerLibraryWithClasses(SharedLibraryLoader.java:74)
        at co.libly.resourceloader.SharedLibraryLoader.load(SharedLibraryLoader.java:57)
        at com.goterl.lazycode.lazysodium.utils.LibraryLoader.loadBundledLibrary(LibraryLoader.java:133)
        at com.goterl.lazycode.lazysodium.utils.LibraryLoader.loadLibrary(LibraryLoader.java:94)
        at com.goterl.lazycode.lazysodium.SodiumJava.<init>(SodiumJava.java:34)
        at com.goterl.lazycode.lazysodium.SodiumJava.<init>(SodiumJava.java:23)
        at mdnet.base.Main.main(Main.kt:94)
        Suppressed: java.lang.UnsatisfiedLinkError: The specified module could not be found.

                at com.sun.jna.Native.open(Native Method)
                at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:191)
                ... 9 more
        Suppressed: java.lang.UnsatisfiedLinkError: The specified module could not be found.

                at com.sun.jna.Native.open(Native Method)
                at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:204)
                ... 9 more
        Suppressed: java.io.IOException: Native library (win32-x86-64/C:\Users\Arnold\AppData\Local\Temp\resource-loader15060737488612266253\windows64\libsodium.dll) not found in resource path (test.jar)
                at com.sun.jna.Native.extractFromResourcePath(Native.java:1095)
                at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:276)
                ... 9 more
gurpreet- commented 4 years ago

@carbotaniuman how do you initialise Lazysodium? Do you have some code you could share?

Looking at it closely why does your path start with win32-x86-64/C:\Users\....? win32-x86-64/ is completely wrong at the start. If you just used C:\Users\Arnold\AppData\Local\Temp\resource-loader15060737488612266253\windows64\libsodium.dll then it would work.

Are you supplying a path or doing anything extra?

carbotaniuman commented 4 years ago

No path, just 'LazySodiumJava lazySodium = new LazySodiumJava(new SodiumJava());'

This is not related to the space in path issue btw.

gurpreet- commented 4 years ago

Could you try:

  1. LazySodiumJava lazySodium = new LazySodiumJava(new SodiumJava(LibraryLoader.Mode.BUNDLED_ONLY));
  2. LazySodiumJava lazySodium = new LazySodiumJava(new SodiumJava(LibraryLoader.Mode.SYSTEM_ONLY));

and post both results/stacktraces here?

gurpreet- commented 4 years ago

I have just migrated to AzureCI to be able to test Lazysodium in an isolated Windows/Linux/MacOS environment. Previously it was on SemaphoreCI and Lazysodium was only being tested in a Linux environment. Now that it's being tested across different isolated platforms, we will be able to easily point out if it's a problem with the developer's machine or not, so tickets like this will be less of a likelihood.

Please keep raising tickets if you encounter a problem.

gurpreet- commented 4 years ago

@carbotaniuman The tests are passing in an isolated Windows environment, therefore I suspect the issue is something to do with your system. This might help https://stackoverflow.com/questions/49596360/native-library-not-found-in-resource-path

It would seem that you have both 64-bit and 32-bit JRE's installed.

Please open a new ticket if the issue persists and you need more help.