GraalVM master tree fails to build Mandrel with JDK 22 EA on Windows

[jerboaa]

Description

We see a Windows specific build failure of GraalVM master on Windows:

 Building com.oracle.svm.native.jvm.windows_amd64 with Ninja... [dependency svm-jvmfuncs-fallback-builder updated]
Error: could not open `D:\a\mandrel\mandrelvm\lib\svm\bin\native-image.export-list'
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Failed, exit code: 1
    at OperatingSystem.exec(build.java:1821)
    at build.buildAgents(build.java:310)
    at build.main(build.java:253)
Caused by: java.lang.RuntimeException: Failed, exit code: 1
    at OperatingSystem.exec(build.java:1810)
    at build.buildAgents(build.java:310)
    at build.main(build.java:253)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
    at java.base/java.lang.reflect.Method.invoke(Method.java:580)
    at jdk.compiler/com.sun.tools.javac.launcher.Main.execute(Main.java:484)
    at jdk.compiler/com.sun.tools.javac.launcher.Main.run(Main.java:208)
    at jdk.compiler/com.sun.tools.javac.launcher.Main.main(Main.java:135)
Error: could not open `D:\a\mandrel\mandrelvm\lib\svm\bin\native-image.export-list'
Error: Process completed with exit code 1.

See: https://github.com/graalvm/mandrel/actions/runs/6819594271/job/18547716716#step:8:426

The same version worked fine to build on Linux:

INFO [build] Congratulations you successfully built Mandrel 24.0.0-dev01ac2d39 based on Java 22-beta+22-ea

https://github.com/graalvm/mandrel/actions/runs/6819594271/job/18547714537#step:8:614

[Karm]

In my notes from 2023-10-30 jenkins:

Temurin 22 Graal master windows

DEBUG [build] Build native agents...
DEBUG [OperatingSystem] Execute [.\mandrel-java22-24.0.0-dev22adacd5e5f\lib\svm\bin\native-image.cmd, --macro:native-image-agent-library] in C:\workspace\workspace\mandrel-maste---5536ebd7\JDK_RELEASE\ea\JDK_VERSION\22\LABEL\w2k19 with environment variables []
Error: could not open `C:\workspace\workspace\mandrel-maste---5536ebd7\JDK_RELEASE\ea\JDK_VERSION\22\LABEL\w2k19\mandrel-java22-24.0.0-dev22adacd5e5f\lib\svm\bin\native-image.export-list'
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Failed, exit code: 1
    at OperatingSystem.exec(build.java:1821)b
    at build.buildAgents(build.java:310)
    at build.main(build.java:253)
Caused by: java.lang.RuntimeException: Failed, exit code: 1
    at OperatingSystem.exec(build.java:1810)
    at build.buildAgents(build.java:310)
    at build.main(build.java:253)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
    at java.base/java.lang.reflect.Method.invoke(Method.java:580)
    at jdk.compiler/com.sun.tools.javac.launcher.Main.execute(Main.java:484)
    at jdk.compiler/com.sun.tools.javac.launcher.Main.run(Main.java:208)
    at jdk.compiler/com.sun.tools.javac.launcher.Main.main(Main.java:135)
Build step 'Execute Windows batch command' changed build result to UNSTABLE

IMHO still the same issue with unexported symbols? I recall seeing it fail on Linux for some time (better error message though) too and Linux seems fixed now.

[jerboaa]

Error: could not open `C:\workspace\workspace\mandrel-maste---5536ebd7\JDK_RELEASE\ea\JDK_VERSION\22\LABEL\w2k19\mandrel-java22-24.0.0-dev22adacd5e5f\lib\svm\bin\native-image.export-list

What is this file supposed to contain? I don't see it for Linux builds, unless I'm looking in the wrong place...

IMHO still the same issue with unexported symbols?

Care to clarify what that "unexported symbols" problem is/was?

[Karm]

Error: could not open `C:\workspace\workspace\mandrel-maste---5536ebd7\JDK_RELEASE\ea\JDK_VERSION\22\LABEL\w2k19\mandrel-java22-24.0.0-dev22adacd5e5f\lib\svm\bin\native-image.export-list

What is this file supposed to contain? I don't see it for Linux builds, unless I'm looking in the wrong place...

AFAIK it's Windows lib file specific, e.g dumpbin /exports on a .lib file lists exported symbols.

IMHO still the same issue with unexported symbols?

Care to clarify what that "unexported symbols" problem is/was?

I don't have the build any more, but lemme re-run 5536ebd7 on Linux and see.

[stooke]

I was able to compile this successfully on my system, which does use a different build script and does build on the C:\ drive. I know that that's not very helpful, but it's a data point that makes me wonder if something in the build harness has changed? Also, my built Graal native image did in fact have a ./lib/svn/bin/native-image.export-list

[stooke]

Looking at the graal code, I would look at https://github.com/oracle/graal/pull/7557 which added this file. It's python, so simple to add prints and see that the drive letter is passed properly as part of the directory path.

[jerboaa]

I was able to compile this successfully on my system, which does use a different build script and does build on the C:\ drive.

@stooke did that compile use mandrel-packaging?

I'm starting to wonder if that file, native-image.export-list gets copied properly.

https://github.com/graalvm/mandrel-packaging/blob/991bd2e95d8ccc920a0015a4bdb15eaf79eaed8d/build.java#L132-L134

Looking at https://github.com/oracle/graal/pull/7557 it's now a built-time-generated thing, while it used to be a on-first-native-image-invoke-generated thing.

[stooke]

@jerboaa when I tried to use mandrel-packaging (you mean the Java-driven build?), I got an unknown package, something like org.graal.options (I don't remember the specifics). My current plan is to try to reconfigure my VM and build partially off the D drive.

I'm starting to wonder if that file, native-image.export-list gets copied properly.

that's why I asked about the D: drive; often Linux code ported to Windows gets that wrong.

[jerboaa]

@jerboaa when I tried to use mandrel-packaging (you mean the Java-driven build?), I got an unknown package, something like org.graal.options (I don't remember the specifics). My current plan is to try to reconfigure my VM and build partially off the D drive.

I think to reproduce, a mandrel-packaging style build would need to be used like GHA does:

   cmd.exe /c "call `"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat`" && set > %temp%\vcvars64.txt"
  Get-Content "$Env:temp\vcvars64.txt" | Foreach-Object {
    if ($_ -match "^(.*?)=(.*)$") {
      Set-Content "Env:\$($matches[1])" $matches[2]
    }
  }
  Set-Location -Path $Env:MANDREL_PACKAGING_REPO
  & $Env:JAVA_HOME\bin\java -ea build.java `
    --mx-home $Env:MX_PATH `
    --mandrel-repo $Env:MANDREL_REPO `
    --mandrel-home $Env:MANDREL_HOME
  & $Env:MANDREL_HOME\bin\native-image --version
  Remove-Item -Recurse $Env:JAVA_HOME
  Move-Item -Path $Env:MANDREL_HOME -Destination $Env:JAVA_HOME

[jerboaa]

Should be fixed with https://github.com/graalvm/mandrel-packaging/pull/380. I.e. it was indeed a mandrel packaging issue.

[jerboaa]

Closing as fixed. Windows Mandrel builds are ok again.