search

graalvm / native-build-tools

Native-image plugins for various build tools
https://graalvm.github.io/native-build-tools/
Other
349 stars 51 forks source link

Task :metadataCopy fails with `FileNotFoundException`. #550

Closed tglaeser closed 3 months ago

tglaeser commented 7 months ago

Description Executing Gradle task metadataCopy as outlined in Reflection support and running with the native agent fails with Error: Unknown name in option specification: macro:native-image-configure-launcher.

To Reproduce

$ ./gradlew metadataCopy --task run --dir src/main/resources/META-INF/native-image

> Task :metadataCopy FAILED
Error: Unknown name in option specification: macro:native-image-configure-launcher
Available macro options are:
    --macro:native-image-agent-library
    --macro:native-image-diagnostics-agent-library
    --macro:native-image-launcher

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':metadataCopy'.
> Process 'command '/opt/mandrel-java17-23.0.2.1-Final/bin/native-image'' finished with non-zero exit value 20

I'm not doing anything special, just following the above mentioned documentation.

System Info

$ uname -srmo
Linux 5.15.133.1-microsoft-standard-WSL2 x86_64 GNU/Linux
$ native-image --version
native-image 17.0.9 2023-10-17
GraalVM Runtime Environment Mandrel-23.0.2.1-Final (build 17.0.9+9)
Substrate VM Mandrel-23.0.2.1-Final (build 17.0.9+9, serial gc)
plugins {
    id 'java-library'
    id 'org.graalvm.buildtools.native' version '0.9.28'
}

Additional context I wonder if this is related to issue #194 and what the doc says here? If the Native Image Configure Tool is not guaranteed to be available and command

$ native-image --macro:native-image-configure-launcher
Error: Unknown name in option specification: macro:native-image-configure-launcher
Available macro options are:
    --macro:native-image-agent-library
    --macro:native-image-diagnostics-agent-library
    --macro:native-image-launcher

fails, how can the Gradle plugin depend on this functionality?

tglaeser commented 5 months ago

Anyone, any pointers please? I can successfully execute ./gradlew -Pagent run to collect the metadata, but ultimately I want to collect the metadata while executing the tests via ./gradlew -Pagent nativeTest which depends on the above mentioned task. This is critical functionality to me. What am I missing?

tglaeser commented 5 months ago

@melix, @dnestoro - Any pointers from you please? Thanks.

tglaeser commented 4 months ago

Anything wrong with this ticket/question?

fniephaus commented 4 months ago

Hi @tglaeser, sorry for the delayed response. It seems you are using Mandrel, which is a GraalVM distro specifically for Quarkus. Unfortunately, Quarkus is not using the Native Build Tools, and apparently, Mandrel is not tested with them. Could you please try this again with GraalVM CE or Oracle GraalVM?

tglaeser commented 4 months ago

Yes, that makes a difference. Command sequence

$ ./gradlew -Pagent clean test
$ ./gradlew metadataCopy --task test --dir src/main/resources/META-INF/native-image/

works. However

$ ./gradlew -Pagent clean run
$ ./gradlew metadataCopy --task run --dir src/main/resources/META-INF/native-image/

continues to fail, now with error

> Task :metadataCopy FAILED
java.io.FileNotFoundException: .../build/native/agent-output/run/reflect-config.json (No such file or directory)

This is is because the metadata has been written under .../build/native/agent-output/run/session-28305-20240306T222136Z, not .../build/native/agent-output/run.

fniephaus commented 4 months ago

This is is because the metadata has been written under .../build/native/agent-output/run/session-28305-20240306T222136Z, not .../build/native/agent-output/run.

@dnestoro could you please try reproduce and fix this?

dnestoro commented 4 months ago

@dnestoro could you please try reproduce and fix this?

Sure! I will take a look today or tomorrow

melix commented 4 months ago

The launcher is uses to build the tool responsible for merging agent files. If there's no launcher, we cannot merge these.

tglaeser commented 4 months ago

Now that I got your attention, I'm seeing other warnings/NPE's when executing a task with the agent attached:

> Task :test
[0.080s][warning][jni,resolve] Re-registering of platform native method: jdk.internal.misc.Unsafe.allocateInstance(Ljava/lang/Class;)Ljava/lang/Object; from code in a different classloader

Warning: Error processing trace entry map(size=5, {(tracer,reflect),(function,getSystemResource),(caller_class,org.apache.logging.log4j.core.util.Loader),(result,true),(args,[null])}): java.lang.NullPointerException
        at java.base@21.0.2/java.util.regex.Pattern.quote(Pattern.java:1499)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.trace.ReflectionProcessor.processEntry(ReflectionProcessor.java:85)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.trace.TraceProcessor.processEntry(TraceProcessor.java:88)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.tracing.ConfigurationResultWriter.traceEntry(ConfigurationResultWriter.java:53)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.tracing.core.Tracer.traceCall(Tracer.java:117)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.BreakpointInterceptor.traceBreakpoint(BreakpointInterceptor.java:187)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.BreakpointInterceptor.traceReflectBreakpoint(BreakpointInterceptor.java:176)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.BreakpointInterceptor.handleGetSystemResources(BreakpointInterceptor.java:619)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.BreakpointInterceptor.getSystemResource(BreakpointInterceptor.java:609)
        at org.graalvm.nativeimage.agent.tracing/com.oracle.svm.agent.BreakpointInterceptor.onBreakpoint(BreakpointInterceptor.java:1112)

Do you want me to open another ticket for that?

fniephaus commented 4 months ago

Yes, please. Could you open that at https://github.com/oracle/graal/issues?

fniephaus commented 4 months ago

The launcher is uses to build the tool responsible for merging agent files. If there's no launcher, we cannot merge these.

We should probably fail with a better error message in the first place if native-image-configure cannot be found.

This is is because the metadata has been written under .../build/native/agent-output/run/session-28305-20240306T222136Z, not .../build/native/agent-output/run.

@melix this is probably another issue that we should fix.

melix commented 4 months ago

I think it's the same issue. The session-28305-20240306T222136Z directories are created by during the execution of the agent. The reason there is a timestamp is because you can typically have multiple JVMs forked during testing, and each needs to write into its own directory. Then the files are merged, and I think we have this directory kept only because the merge failed.

tglaeser commented 4 months ago

Now, that's interesting. Above I said the Gradle task sequence is

$ ./gradlew -Pagent clean run
$ ./gradlew metadataCopy --task run --dir src/main/resources/META-INF/native-image/

More specifically, after executing the run task I'm hitting Ctrl+C to stop the task. This simply might not be graciously enough to allow the merge action to be executed.

melix commented 4 months ago

mmm, indeed this may be an issue. I don't know what kind of app you are building but maybe adding an endpoint to shutdown the app gracefully would help?

tglaeser commented 4 months ago

Just verified that: Gracefully shutting down the application solves the issue.

tglaeser commented 3 months ago

The issue is understood, closing the ticket.