amd64 shared library loaded on M1 Mac running in Rosetta (needing amd64 libraries)

[thunderbiscuit]

A few of the methods seem to create problems on the native side of things. Here is an example of an error I get:

❯ ./kldk
Hello, thundernode!

Kldk ❯❯❯ startnode
Kldk starting...
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x0000000131a289ad, pid=41586, tid=11523
#
# JRE version: OpenJDK Runtime Environment Temurin-11.0.14+9 (11.0.14+9) (build 11.0.14+9)
# Java VM: OpenJDK 64-Bit Server VM Temurin-11.0.14+9 (11.0.14+9, mixed mode, tiered, compressed oops, g1 gc, bsd-amd64)
# Problematic frame:
# C  [liblightningjni.so+0x1d69ad]  NetworkGraph_new+0xd
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/tb/repos/k-ldk-node/app/build/install/kldk/bin/hs_err_pid41586.log
#
# If you would like to submit a bug report, please visit:
#   https://github.com/adoptium/adoptium-support/issues
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
zsh: abort      ./kldk

To reproduce, simply fire up the kdlk node here on an M1 and run the startnode command.

The error above happens at the networkGraph initialization step, but commenting that out allows us to find similar errors happening on the userConfig or the transactionFilter steps:

# Problematic frame:
# C  [liblightningjni.so+0x272078]  UserConfig_default+0x28

# Problematic frame:
# C  [liblightningjni.so+0x4028c7]  COption_FilterZ_some+0x7

[TheBlueMatt]

@tnull spent a bit of time trying to repro, and wasn't able to. What JVM are you using here?

[thunderbiscuit]

Oh awesome! Just trying out a different JVM actually fixed the issue. I guess that's both good and bad for you though, since it's harder to reproduce I guess.

I was using Temurin 11.0.14 (OpenJDK 64-Bit Server VM Temurin-11.0.14+9), and simply switching to Temurin 17.0.2 fixed my problem right up. Thanks a bunch.

I can close the issue, unless you'd like to keep it live to keep digging. I'm happy to try and help debugging like we did on Discord last week, or to send you the extra long error file that gets printed when the errors happen.

[TheBlueMatt]

Huh! That's....strange, but glad it works? Is the Temurin 11.0.14 release actually x86_64 running with rosetta or is it native arm64?

[thunderbiscuit]

Oh good question.

Turns out my 11.0.4 is an x86_64 so must have been running on Rosetta, whereas my 17.0.2 is arm64.

❯ file /Users/tb/.sdkman/candidates/java/17.0.2-tem/bin/java
/Users/tb/.sdkman/candidates/java/17.0.2-tem/bin/java: Mach-O 64-bit executable arm64

❯ file /Users/tb/.sdkman/candidates/java/11.0.14-tem/bin/java
/Users/tb/.sdkman/candidates/java/11.0.14-tem/bin/java: Mach-O 64-bit executable x86_64

[thunderbiscuit]

It gets confusing because whether you download the right or the wrong JDK for your architecture it adapts by running it on Rosetta and you don't even notice... Anyway that's probably where the problems stem from.

[TheBlueMatt]

Ah! Okay, so I guess then we incorrectly load the amd64 library on x86 (well, rosetta) and that causes a crash, okay, well at least we (probably) know the bug, then.

[ovitrif]

For me they don't work with either jvm architecture (x86_64 / aarch64)…

And it seems to be an issue with binding to a couple of native methods missing init:

java.lang.UnsatisfiedLinkError: 'void org.ldk.enums.Network.init()'
    at org.ldk.enums.Network.init(Native Method)
    at org.ldk.enums.Network.<clinit>(Network.java:24)
    at lightning.NetworkGraphKt.<clinit>(NetworkGraph.kt:6)

NetworkGraph

package lightning

import org.ldk.enums.Network
import org.ldk.structs.NetworkGraph

val network = Network.LDKNetwork_Regtest
val networkGraph = NetworkGraph.of(network, logger)

JDK 0.0.118.0 in build.gradle.kts

implementation(fileTree("libs") { include("*.jar") })

[ovitrif]

@TheBlueMatt Do you think the above is rather a separate issue? I'll provide more details and file a new one if you confirm 👍🏻

[TheBlueMatt]

Hey, sorry for the delay. Can you check that the .nativelib file(s) from the original LDK jar are being included in your built JAR? I'm not sure that gradle does that by default.

[ovitrif]

Hey, sorry for the delay. Can you check that the .nativelib file(s) from the original LDK jar are being included in your built JAR? I'm not sure that gradle does that by default.

Yeah, they are included. It's more likely the same issue pointed in discord under ldk-bindings. It works on Android though via the .aar, but not in Java-only environment, via jar 🤷🏻 .

[TheBlueMatt]

Ah, okay, your issue was unrelated, then, tracking as #149

[ovitrif]

@TheBlueMatt Clear, that makes sense! Thanks a lot for the quick update and the pending fix 🚀 The change makes sense, and it would help a lot on the JVM side, especially for newcomers with that background 🙌🏻