search

substrait-io / substrait-java

Apache License 2.0
72 stars 70 forks source link

[Isthmus][CLI][GraalVM] Failed to build Isthmus native image locally using GraalVM 22.0.0.2 #211

Closed davisusanibar closed 6 months ago

davisusanibar commented 6 months ago

Issue

Currently Isthmus is using GraalVM 22.0.0.2 https://github.com/substrait-io/substrait-java/blob/main/isthmus/build.gradle.kts#L104

It is necessary to run ./gradlew nativeImage in order to build Isthmus native image, and the process will finish with the following message:

./gradlew nativeImage

> Task :isthmus:extractGraalTooling
Downloading: Release index file from oca.opensource.oracle.com
Error: Error reading component list: https://oca.opensource.oracle.com/gds/meta-data.json

> Task :isthmus:extractGraalTooling FAILED

Build substrait FAILURE reason:                                
    Execution failed for task ':isthmus:extractGraalTooling':
        org.gradle.process.internal.ExecException: Process 'command '/Users/dsusanibar/.gradle/caches/com.palantir.graal/22.0.0.2/17/graalvm-ce-java17-22.0.0.2/Contents/Home/bin/gu'' finished with non-zero exit value 3

Reason of the error

Isthmus is using Palantir Gradle Graal to download and locally cache a GraalVM installation and make available select parts of the GraalVM compiler for use in Gradle builds.

GraalVM 22.0.0.2 is available to download, but not to install a new image, as you can see at:

$ pwd
/Users/dsusanibar/.gradle/caches/com.palantir.graal/22.0.0.2/17/graalvm-ce-java17-22.0.0.2/Contents/Home/bin

$ ./gu --version
GraalVM Updater 22.0.0.2

$ ./gu available
Downloading: Release index file from oca.opensource.oracle.com
Error: Error reading component list: https://oca.opensource.oracle.com/gds/meta-data.json

$ ./gu install native-image
Downloading: Release index file from oca.opensource.oracle.com
Error: Error reading component list: https://oca.opensource.oracle.com/gds/meta-data.json

Workaround

$ pwd
/Users/dsusanibar/.gradle/caches/com.palantir.graal/22.1.0/17/graalvm-ce-java17-22.1.0/Contents/Home/bin
$ ./gu --version
GraalVM Updater 22.1.0
$ bin ./gu available
Downloading: Component catalog from www.graalvm.org
ComponentId              Version             Component name                Stability                     Origin
---------------------------------------------------------------------------------------------------------------------------------
espresso                 22.1.0              Java on Truffle               Experimental                  github.com
espresso-llvm            22.1.0              Java on Truffle LLVM Java librExperimental                  github.com
llvm-toolchain           22.1.0              LLVM.org toolchain            Supported                     github.com
native-image             22.1.0              Native Image                  Early adopter                 github.com
nodejs                   22.1.0              Graal.nodejs                  Supported                     github.com
python                   22.1.0              Graal.Python                  Experimental                  github.com
R                        22.1.0              FastR                         Experimental                  github.com
ruby                     22.1.0              TruffleRuby                   Experimental                  github.com
wasm                     22.1.0              GraalWasm                     Experimental                  github.com
$ ./gu install native-image
Downloading: Component catalog from www.graalvm.org
Processing Component: Native Image
Component Native Image (org.graalvm.native-image) is already installed.
Error: Classes that should be initialized at run time got initialized during image building:
 org.apache.calcite.util.BuiltInMethod was unintentionally initialized at build time. To see why org.apache.calcite.util.BuiltInMethod got initialized use --trace-class-initialization=org.apache.calcite.util.BuiltInMethod

Consider: You won't experience any problems if your local environment just caches the GraalVM image at the moment when the native image was able to download and install (e.g. current Github CI Jobs).

vibhatha commented 6 months ago

@vbarua could we upgrade the GraalVM or is there an alternative for this?

vbarua commented 6 months ago

Upgrading seems like a reasonable way to address this. I'd be happy to review a PR for an upgrade to the latest version that works with no changes. We can file an issue to move us to the latest version and deal with the changes required for that.

vibhatha commented 6 months ago

@vbarua we got one PR merged related to a related update. What more work would we need to close this?

vbarua commented 6 months ago

Isthmus native image builds now run successfully using GraalVM 22.1.0.

Opened a separate issue to upgrade to the latest version, though I don't think it's critical to do so: https://github.com/substrait-io/substrait-java/issues/218