"No error" crashes when running AWT functions on MacOS

[yairm210]

Hi, looks like a great possible replacement for packr!

Unfortunately, when I tried adding this to my Kotlin project, it bundled fine but then I got the following:

thread 'main' panicked at src/main.rs:140:6:
Failed to call main method: JavaException
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Exception in thread "Thread-0" java.lang.NoClassDefFoundError: kotlin/jvm/internal/Intrinsics
        at com.unciv.app.desktop.DesktopLauncher.main(DesktopLauncher.kt)
Caused by: java.lang.ClassNotFoundException: kotlin.jvm.internal.Intrinsics
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source)
        at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
        ... 1 more

So it looks like it's missing some basic files.

When I try to run the jar directly I get

❯ java -jar desktop-1.0.1.jar
no main manifest attribute, in desktop-1.0.1.jar

In order to build my .jar file that works, I use the following gradle config, which has a lot of little changes picked up over the years :

tasks.register<Jar>("dist") { // Compiles the jar file
    dependsOn(tasks.getByName("classes"))

    // META-INF/INDEX.LIST and META-INF/io.netty.versions.properties are duplicated, but I don't know why
    duplicatesStrategy = DuplicatesStrategy.EXCLUDE

    from(files(sourceSets.main.get().output.resourcesDir))
    from(files(sourceSets.main.get().output.classesDirs))
    // see Laurent1967's comment on https://github.com/libgdx/libgdx/issues/5491
    from({
        (
            configurations.runtimeClasspath.get().resolve() // kotlin coroutine classes live here, thanks https://stackoverflow.com/a/59021222
            + configurations.compileClasspath.get().resolve()
        ).map { if (it.isDirectory) it else zipTree(it) }})
    from(files(assetsDir))
    exclude("mods", "SaveFiles", "MultiplayerFiles", "GameSettings.json", "lasterror.txt")
    // This is for the .dll and .so files to make the Discord RPC work on all desktops
    from(files(discordDir))
    archiveFileName.set("${BuildConfig.appName}.jar")

    manifest {
        attributes(mapOf("Main-Class" to mainClassName, "Specification-Version" to BuildConfig.appVersion))
    }
}

Config:

plugins {
    id("kotlin")
    id("io.github.fourlastor.construo") version "1.2.0"
}
construo {
    // Construo fields should have docs
    // name of the executable
    name.set(BuildConfig.appName.lowercase())
    // human-readable name, used for example in the `.app` name for macOS
    humanName.set(BuildConfig.appName)
    // Optional, defaults to project version
    version.set(appVersion)
    // Optional, defaults to application.mainClass or jar task main class
    mainClass.set(mainClassName)
    // Optional, defaults to $buildDir/construo/dist
    // where to put the packaged zips
//        outputDir.set(deployFolder)

    targets{
        create<Target.MacOs>("macM1") {
            architecture.set(Target.Architecture.AARCH64)
            jdkUrl.set("https://github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.11%2B9/OpenJDK17U-jdk_aarch64_mac_hotspot_17.0.11_9.tar.gz")
            identifier.set(BuildConfig.identifier)
        }
    }
}

Attached is the M1 zip created

In fact what would be great for me is if I could tell it "here's the jar file, don't worry about it, just pack"

The way I can with packr :)

[fourlastor]

hi! this doesn't depend on Kotlin, I believe it's because construo is not aware right now of other jar tasks other than jar and shadowJar (and it's using the task jar in your case, which hasn't been configured). I think you could solve it by changing:

tasks.register<Jar>("dist") {
  // configuration here
}

with

tasks.jar {
    // configuration here
}

I don't have plans currently to support non-standard jar tasks, this might change in the future, but definitely not soon

Another thing I do suggest to take a look at is shadow, which produces a "fat jar", as in a jar with all the dependencies inlined inside

[yairm210]

That works, the jar used is now correct, the wrapper now fails with no real info 🤔

~/dev/unciv/desktop/build/construo/macM1/Unciv.app/Contents/MacOS master* 0s                                                                                                             17/07/24 16:04:25
❯ ./unciv
[1]    63526 trace trap  ./unciv

[fourlastor]

Try to run it with RUST_BACKTRACE=full before the command.

Does the jar work if launched with java -jar <filename>.jar?

[yairm210]

The jar works

~/dev/unciv/desktop/build/construo/macM1/Unciv.app/Contents/MacOS master* 0s                                                                                                             17/07/24 16:08:32
❯ RUST_BACKTRACE=full ./unciv
[1]    63770 trace trap  RUST_BACKTRACE=full ./unciv

No difference :/

But now I notice it never even created the dist folder, I'm running directly from construo/macM1/Unciv.app/Contents/MacOS

[fourlastor]

This looks like a jdk minification issue, does it work if you replace the jdk folder in the same folder as the executable with the original url you set in the gradle config to download it? If that works, it's definitely an issue when minifying the JDK (which is particularly tricky as the modules your app needs aren't always obvious, not even using jdeps from the JDK)

Things I would look at are:

Are you loading classes from the stdlib via reflection? In that case add

construo {
    jlink {
        // add arbitrary modules to be included when running jlink
        modules.addAll("module.name.here")
    }
}

Where module.name.here is the module containing that specific class (they're specified in the documentation from Oracle).

If you're doing HTTPS requests, that might be because of crypto modules missing, then you should be able to fix it by adding this:

construo {
    jlink {
        // add arbitrary modules to be included when running jlink
        modules.addAll("jdk.crypto.ec")
    }
}

I haven't found yet a less trial and error way of solving this issue, if you push a branch with the commits somewhere I can take a look when I have some time (I'll try tomorrow or Friday)

[yairm210]

Even copying the entire JDK doesn't solve this :(

Here's the branch, run gradlew desktop:packageMacM1 from the root folder

The gradle file is desktop/build.gradle.kts

Take your time, I know how these things are, I'm in no rush :)

[fourlastor]

I tried packaging a Linux x64 version (I don't have a Mac) and it runs, so at least we know it isn't a minification issue.

I think this might be an issue in roast (which is the runtime construo uses to startup the jdk) specific for MacOS. One way of getting a better error output could be running roast from source, it's a few steps but it shouldn't be complex:

1. Package the app with construo
2. clone roast (tag 0.0.7, the newest version changed a bit of the api)
3. Copy these files from the packaged game to the root folder of roast: config.json, jdk folder, Unciv.jar
4. Run roast from source (I use Visual Studio Code with the rust extension by Microsoft, but there's also RustRover from Jetbrains which is free for non-commercial projects, so you can use it with roast)

Note that you might need to install rust, I had for vscode. I can also try but it might take a while until I get access to a Mac

[yairm210]

I'll try it! And I'll try packaging a Linux version, if it works fine on Windows and Linux I'll move to it even if the Mac stuff is off :)

[fourlastor]

Hello! Did you have a chance to test this?

[yairm210]

Hi, sorry My son was born 2 weeks ago so that's been taking up most of my time :) Hope to get to this soon 🙏🏿

[fourlastor]

Congrats! And no hurry 😁

[yairm210]

Got the same thing with running roast locally

Managed to set up a vscode debugger with lldb and got this far:

So it recognized the return type was jni::wrapper::signature::Primitive::Void and thus activated the Void function, and then it suddenly died

The macro expands to

which does have a log::trace which we don't seem to be getting to 🤔

[fourlastor]

Hmm, that's my ignorance in Rust apparently :laughing: log doesn't log anything unless you specify a logger. I created a branch which adds a logger, you can get all logs by running your application with RUST_LOG=trace as an environment variable.

I think you might be able to try directly with the built binaries here by running RUST_LOG=trace ./roast (roast should be in your project folder as usually, with the jdk, config, and jar files).

[yairm210]

❯ RUST_LOG=trace ./roast
[2024-07-31T11:55:41Z TRACE jni::wrapper::java_vm::vm] calling unchecked JavaVM method: DetachCurrentThread
[2024-07-31T11:55:41Z TRACE jni::wrapper::java_vm::vm] looking up JavaVM method DetachCurrentThread
[2024-07-31T11:55:41Z TRACE jni::wrapper::java_vm::vm] found JavaVM method
[2024-07-31T11:55:41Z TRACE jni::wrapper::java_vm::vm] calling unchecked JavaVM method: GetEnv
[2024-07-31T11:55:41Z TRACE jni::wrapper::java_vm::vm] looking up JavaVM method GetEnv
[2024-07-31T11:55:41Z TRACE jni::wrapper::java_vm::vm] found JavaVM method
[2024-07-31T11:55:41Z TRACE jni::wrapper::java_vm::vm] calling unchecked JavaVM method: AttachCurrentThread
[2024-07-31T11:55:41Z TRACE jni::wrapper::java_vm::vm] looking up JavaVM method AttachCurrentThread
[2024-07-31T11:55:41Z TRACE jni::wrapper::java_vm::vm] found JavaVM method
[2024-07-31T11:55:41Z DEBUG jni::wrapper::java_vm::vm] Attached thread main (ThreadId(1)). 1 threads attached
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling checked jni method: NewStringUTF
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method NewStringUTF
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] checking for exception
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling unchecked jni method: ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] no exception found
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling checked jni method: NewStringUTF
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method NewStringUTF
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] checking for exception
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling unchecked jni method: ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] no exception found
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling checked jni method: FindClass
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method FindClass
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] checking for exception
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling unchecked jni method: ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] no exception found
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling checked jni method: NewObjectArray
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method NewObjectArray
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] checking for exception
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling unchecked jni method: ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] no exception found
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling unchecked jni method: DeleteLocalRef
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method DeleteLocalRef
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling checked jni method: SetObjectArrayElement
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method SetObjectArrayElement
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] checking for exception
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling unchecked jni method: ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] no exception found
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling checked jni method: FindClass
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method FindClass
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] checking for exception
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling unchecked jni method: ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] no exception found
[2024-07-31T11:55:41Z TRACE jni::wrapper::objects::jvalue] converted Object(JObject { internal: 0x12d610cd8, lifetime: PhantomData<&()> }) to jvalue 5056302296
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling checked jni method: GetStaticMethodID
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method GetStaticMethodID
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] checking for exception
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling unchecked jni method: ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method ExceptionCheck
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] no exception found
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] calling checked jni method: CallStaticVoidMethodA
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] looking up jni method CallStaticVoidMethodA
[2024-07-31T11:55:41Z TRACE jni::wrapper::jnienv] found jni method
[1]    16253 trace trap  RUST_LOG=trace ./roast

So we managed to get to an even inner-er macro in jni, jni_method, and we come to actually activate that method, and then boom

macro_rules! jni_method {
    ( $jnienv:expr, $name:tt ) => {{
        log::trace!("looking up jni method {}", stringify!($name));
        let env = $jnienv;
        match deref!(deref!(env, "JNIEnv"), "*JNIEnv").$name {
            Some(method) => {
                log::trace!("found jni method");
                method
            }
            None => {
                log::trace!("jnienv method not defined, returning error");
                return Err($crate::errors::Error::JNIEnvMethodNotFound(stringify!(
                    $name
                )));
            }
        }
    }};
}

So I tried a different approach. When I replaced my main function with just "hello world" everything worked, so it's something within the function that's causing this So while this is something that shouldn't silently fail, it is something that is code-specific to me 🤔

I gradually managed to isolate the problem to a specific Java function - GraphicsEnvironment.getLocalGraphicsEnvironment() based on java.awt.GraphicsEnvironment

Rechecked that it wasn't a JDK issue

SO APPARENTLY it's a MacOS + AWT issue

Perhaps related to https://github.com/oracle/graal/issues/4124 - it looks like AWT libraries have some magic going on

[fourlastor]

Aha! Great work on digging deeper on this :)

Maybe you could try packing with Liberica https://bell-sw.com/pages/downloads/#jdk-17-lts ? In the linked issue comments they say that the issue doesn't happen with it (even tho they're using the native image, so this might not work)

[yairm210]

Unfortunately that does not solve the problem :( Tried running the jar from both jdks in the command line (jdk/bin/java -jar Unciv.jar) and it Just Works so I have no idea what linking nonsense is going on here Yet another exciting Java Runtime Experience ™️ adventure

Well, it was worth a shot 🤷🏿

[fourlastor]

You could try to sorround the call in try/catch and see if you manage to get a stack trace:

try {
  GraphicsEnvironment.getLocalGraphicsEnvironment() 
} catch (t: Throwable) {
  // print t's stacktrace
}

[fourlastor]

Another test you could do is to remove this dependency: https://github.com/yairm210/Unciv/blob/efb21182dba44da4351ba2d93c2341f6e4fb1ff1/desktop/build.gradle.kts#L30, I had problems in the past with things that would try to fix macos not running java on the first thread

[yairm210]

The try/catch yielded nothing but removing the dependency did change things

The AWT no longer fails, but we just get a blank screen with default background color. When running regularly (java -jar) we now (expectedly) get the GLFW may only be used on the main thread and that thread must be the first thread in the process. Please run the JVM with -XstartOnFirstThread. This check may be disabled with Configuration.GLFW_CHECK_THREAD0. error

I think I'll stop here though - I'm nearing the end of my hopefulness with this 😓 So to summarize - it seems to have to do with AWT and the MacOS "run on first thread"

It's not you, I personally don't even blame the JVM ecosystem which I feel is not converging towards helpfulness rather than away. Instead, I blame Apple for design decisions which are geared towards them delivering a holistic Apple experience at the cost of making things more difficult for developers 😓 I'm not sure I would make different decisions if I were them, doesn't mean I like the results -_-

---

Unrelatedly (probably) I encountered a different issue with Native.load which I use to notify the local Discord that 'game X is playing':

WARNING: JNI local refs: 33, exceeds capacity: 32
    at jdk.internal.loader.NativeLibraries.load(java.base@17.0.11/Native Method)
    at jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(java.base@17.0.11/NativeLibraries.java:388)
    at jdk.internal.loader.NativeLibraries.loadLibrary(java.base@17.0.11/NativeLibraries.java:232)
    - locked <0x00002000000a9858> (a java.util.HashSet)
    at jdk.internal.loader.NativeLibraries.loadLibrary(java.base@17.0.11/NativeLibraries.java:174)
    at java.lang.ClassLoader.loadLibrary(java.base@17.0.11/ClassLoader.java:2394)
    at java.lang.Runtime.load0(java.base@17.0.11/Runtime.java:755)
    at java.lang.System.load(java.base@17.0.11/System.java:1957)
    at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1045)
    at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:1015)
    at com.sun.jna.Native.<clinit>(Native.java:221)
    at com.unciv.app.desktop.DiscordUpdater.startUpdates(DiscordUpdater.kt:33)
    at com.unciv.app.desktop.DesktopGame.<init>(DesktopGame.kt:33)
    at com.unciv.app.desktop.DesktopLauncher.main(DesktopLauncher.kt:104)
WARNING in native method: JNI call made without checking exceptions when required to from CallStaticObjectMethod
    at jdk.internal.loader.NativeLibraries.load(java.base@17.0.11/Native Method)
    at jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(java.base@17.0.11/NativeLibraries.java:388)
    at jdk.internal.loader.NativeLibraries.loadLibrary(java.base@17.0.11/NativeLibraries.java:232)
    - locked <0x00002000000a9858> (a java.util.HashSet)
    at jdk.internal.loader.NativeLibraries.loadLibrary(java.base@17.0.11/NativeLibraries.java:174)
    at java.lang.ClassLoader.loadLibrary(java.base@17.0.11/ClassLoader.java:2394)
    at java.lang.Runtime.load0(java.base@17.0.11/Runtime.java:755)
    at java.lang.System.load(java.base@17.0.11/System.java:1957)
    at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1045)
    at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:1015)
    at com.sun.jna.Native.<clinit>(Native.java:221)
    at com.unciv.app.desktop.DiscordUpdater.startUpdates(DiscordUpdater.kt:33)
    at com.unciv.app.desktop.DesktopGame.<init>(DesktopGame.kt:33)
    at com.unciv.app.desktop.DesktopLauncher.main(DesktopLauncher.kt:104)

This is probably not a JDK-specific issue since it replicates with Corretto as well And this is despite it being in a try-catch for throwable :/

[fourlastor]

The warning you get is nothing to worry about, it's because roast adds this flag to log JNI warnings

[fourlastor]

Thanks for testing so much with this! I'll see if I ever get around this issue