search

oracle / graal

GraalVM compiles Java applications into native executables that start instantly, scale fast, and use fewer compute resources 🚀
https://www.graalvm.org
Other
20.4k stars 1.64k forks source link

Create custom runtime using jlink with support for Javascript (or other language) #4572

Closed jamalsa closed 11 months ago

jamalsa commented 2 years ago

Feature request

Please include the following information:

Is your feature request related to a problem? Please describe. I tried to create a custom runtime using jlink with javascript support but unsuccessful so far. The resulting runtime doesn't contain the js engine and when I run my application it return error A language with id 'js' is not installed. Installed languages are: []. For information I'm using default GraalVM distribution which contains js.

Describe the solution you'd like. A way to include additional languages into runtime using jlink (or any other solution).

Describe who do you think will benefit the most. The user who want to use GraalVM to replace Hotspot and support other language without creating native-image.

oubidar-Abderrahim commented 2 years ago

Hi, thank you for reporting this, could you please provide a reproducer to illustrate this issue?

rjbaucells commented 1 year ago

I am having the same problem, steps to reproduce:

openjdk version "17.0.6" 2023-01-17
OpenJDK Runtime Environment GraalVM CE 22.3.1 (build 17.0.6+10-jvmci-22.3-b13)
OpenJDK 64-Bit Server VM GraalVM CE 22.3.1 (build 17.0.6+10-jvmci-22.3-b13, mixed mode, sharing)
import org.graalvm.polyglot.*;
import org.graalvm.polyglot.proxy.*;

public class HelloPolyglot {
    public static void main(String[] args) {
        System.out.println("Hello Java!");
        try (Context context = Context.create()) {
            context.eval("js", "print('Hello JavaScript!');");
        }
    }
}
Hello Java!
Hello JavaScript!
jlink --add-modules "$(java --list-modules | cut -f1 -d'@' | tr '\n' ',')" --output jdk
./jdk/bin/java HelloPolyglot
Hello Java!
Exception in thread "main" java.lang.IllegalArgumentException: A language with id 'js' is not installed. Installed languages are: [].
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotEngineException.illegalArgument(PolyglotEngineException.java:131)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotEngineImpl.throwNotInstalled(PolyglotEngineImpl.java:1109)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotEngineImpl.requirePublicLanguage(PolyglotEngineImpl.java:1116)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.requirePublicLanguage(PolyglotContextImpl.java:1491)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.lookupLanguageContext(PolyglotContextImpl.java:1453)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.eval(PolyglotContextImpl.java:1462)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextDispatch.eval(PolyglotContextDispatch.java:63)
    at org.graalvm.sdk/org.graalvm.polyglot.Context.eval(Context.java:399)
    at org.graalvm.sdk/org.graalvm.polyglot.Context.eval(Context.java:425)
    at HelloPolyglot.main(HelloPolyglot.java:8)

FIX: include languages folder:

cp -R /path/to/graalvm/languages ./jdk/
APXEOLOG commented 1 year ago

I have the same issue with Graal 22.3.1, Java 19. I am using a gradle plugin to build a runtime

runtime {
    options.set(listOf("--strip-debug", "--compress", "1", "--no-header-files", "--no-man-pages"))
    modules.set(
        listOf(
            "java.base", "java.datatransfer", "java.desktop", "java.management", "java.net.http",
            "java.prefs", "java.scripting", "java.sql", "java.xml", "jdk.dynalink", "jdk.unsupported"
        )
    )
}

Output runtime:

openjdk version "19.0.2" 2023-01-17
OpenJDK Runtime Environment GraalVM CE 22.3.1 (build 19.0.2+7-jvmci-22.3-b12)
OpenJDK 64-Bit Server VM GraalVM CE 22.3.1 (build 19.0.2+7-jvmci-22.3-b12, mixed mode)

I also tried to use fix mentoned above (include languages folder). No matter what, I have the following output when running my program:

[To redirect Truffle log output to a file use one of the following options:
* '--log.file=<path>' if the option is passed using a guest language launcher.
* '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher.
* Configure logging using the polyglot embedding API.]
[engine] WARNING: The polyglot context is using an implementation that does not support runtime compilation.
The guest application code will therefore be executed in interpreted mode only.
Execution only in interpreted mode will strongly impact the guest application performance.
For more information on using GraalVM see https://www.graalvm.org/java/quickstart/.
To disable this warning the '--engine.WarnInterpreterOnly=false' option or use the '-Dpolyglot.engine.WarnInterpreterOnly=false' system property.
java.lang.IllegalArgumentException: A language with id 'js' is not installed. Installed languages are: [].
        at com.oracle.truffle.polyglot.PolyglotEngineException.illegalArgument(PolyglotEngineException.java:131)
        at com.oracle.truffle.polyglot.PolyglotEngineImpl.throwNotInstalled(PolyglotEngineImpl.java:1109)
        at com.oracle.truffle.polyglot.PolyglotEngineImpl.requirePublicLanguage(PolyglotEngineImpl.java:1116)
        at com.oracle.truffle.polyglot.PolyglotContextImpl.requirePublicLanguage(PolyglotContextImpl.java:1491)
        at com.oracle.truffle.polyglot.PolyglotContextImpl.lookupLanguageContext(PolyglotContextImpl.java:1453)
        at com.oracle.truffle.polyglot.PolyglotContextImpl.getBindings(PolyglotContextImpl.java:1201)
        at com.oracle.truffle.polyglot.PolyglotContextDispatch.getBindings(PolyglotContextDispatch.java:98)
        at org.graalvm.polyglot.Context.getBindings(Context.java:560)
kaja-mohideen commented 1 year ago

Any update on this issue?

APXEOLOG commented 1 year ago

I was able to create a workaround

// Configure jlink
runtime {
    options.set(mutableListOf("--strip-debug", "--compress", "1", "--no-header-files", "--no-man-pages"))
    modules.set(
        listOf(
            // Application dependencies
            "java.base", "java.datatransfer", "java.desktop", "java.management", "java.net.http", "jdk.crypto.ec",
            "java.prefs", "java.scripting", "java.sql", "java.xml", "jdk.dynalink", "jdk.unsupported",

            // Graal.js requirements
            "org.graalvm.js.scriptengine", "org.graalvm.locator", "org.graalvm.sdk",
            "org.graalvm.truffle", "com.oracle.graal.graal_enterprise", "com.oracle.truffle.regex"
        )
    )
}

// Download a proper graal toolchain to build image for windows
tasks["jre"].doFirst {
    // Setup target platform depending on the current OS
    if (org.gradle.internal.os.OperatingSystem.current().isWindows) {
        // For windows we just use the existing toolchain
        runtime.targetPlatform("windows", getCurrentToolchainLocation().toString())
    } else {
        // For other OS we need to download windows toolchain and use it
        val windowsToolchainLink = org.gradle.toolchains.foojay.FoojayApi().toUri(
            java.toolchain.languageVersion.get(),
            java.toolchain.vendor.get(),
            java.toolchain.implementation.get(),
            OperatingSystem.WINDOWS,
            Architecture.X86_64,
        ).toString()
        val jdkArchiveFolder = project.gradle.gradleUserHomeDir.resolve("jdks")
        val jdkOutputFolder = jdkArchiveFolder.resolve("graalvm_community-19-amd64-windows")
        val jdkArchive = jdkArchiveFolder.resolve("graalvm_community-19-amd64-windows.zip")

        if (!jdkOutputFolder.exists()) {
            println("Downloading windows toolchain from $windowsToolchainLink")
            download {
                src(windowsToolchainLink)
                dest(jdkArchive)
            }
            println("Unpacking windows toolchain into $jdkOutputFolder")
            unzipTo(jdkOutputFolder, jdkArchive)
        } else {
            println("Windows toolchain is already present")
        }

        val windowsToolchainLocation = jdkOutputFolder.listFiles().first().toString()
        println("Windows toolchain location: $windowsToolchainLocation")

        runtime.targetPlatform("windows", windowsToolchainLocation)
    }
}

tasks["jre"].doLast {
    // After the jre is prepared we want to do some operations
    val runtimePath = "$buildDir/jre/hafen-replica-windows"

    delete("$runtimePath/legal") // We don't need legal info
    delete(fileTree("$runtimePath/lib") { // Cleanup remaining .so files
        include("**/*.so")
    })
    delete(fileTree("$runtimePath/bin") { // Cleanup additional .exe files
        include("**/*.exe")
        exclude("java.exe")
    })

    // Now we want to install GraalVM.js backend and copy it to the runtime
    exec {
        executable(getCurrentToolchainLocation().dir("lib/installer/bin")
            .file(if (org.gradle.internal.os.OperatingSystem.current().isWindows) "gu.exe" else "gu"))
        args("install", "js")
    }
    copy {
        into("$runtimePath/languages")
        from(getCurrentToolchainLocation().dir("languages"))
    }
}

fun getCurrentToolchainLocation(): Directory {
    return javaToolchains.compilerFor {
        languageVersion.set(java.toolchain.languageVersion)
        vendor.set(java.toolchain.vendor)
    }.get().metadata.installationPath
}

fun Task.download(action: de.undercouch.gradle.tasks.download.DownloadAction.() -> Unit) =
    download.configure(delegateClosureOf(action))
pl71 commented 1 year ago

I have the same issue in Eclipse. Can't apply fix by @rjbaucells Waiting for any updates.

https://www.graalvm.org/java/quickstart/ - Page not found :-(

chumer commented 11 months ago

Jlink is now properly supported with 23.1 and Truffle Unchained. Our sample repository comes with a configuration: https://github.com/graalvm/polyglot-embedding-demo/blob/main/pom.xml#L133

This should do the trick for Maven:

            <plugin>
                <artifactId>maven-jlink-plugin</artifactId>
                <version>3.1.0</version>
                <extensions>true</extensions>
                <configuration>
                    <ignoreSigningInformation>true</ignoreSigningInformation>
                </configuration>
            </plugin>
kaja-mohideen commented 11 months ago

@chumer It would be very helpful if there is some documentation capturing the graal/truffle related modules to use with jlink for people who don't use Maven.

chumer commented 11 months ago

@kaja-mohideen you no longer need to do that (with 23.1). The Jlink plugin (I am sure there is one for Gradle too) derives that from your dependencies.

mikehearn commented 10 months ago

@kaja-mohideen If you need an explicit list for some reason, here's an example set of modules needed for GraalPy expressed in the form of Conveyor configuration, it should be obvious enough:

  jvm.modules = ${app.jvm.modules} [
    "com.oracle.graal.graal_enterprise"
    "com.oracle.truffle.{enterprise,regex,truffle_nfi,truffle_nfi_libffi,tools.profiler}"
    "org.graalvm.{word,polyglot,collections,jniutils,nativebridge,launcher}",
    "org.bouncycastle.{pkix,provider,util}"
    "org.graalvm.truffle"
    "org.graalvm.truffle.{compiler,runtime}"
    "org.graalvm.shadowed.{icu4j,jline,org.json}"

    // Python stuff
    "org.graalvm.py"
    "org.graalvm.py.{enterprise,launcher,resources}"
    "org.tukaani.xz"
    "org.graalvm.llvm.api"
  ]
  jvm.options = ${app.jvm.options} [
    "-XX:+UnlockExperimentalVMOptions"
    "-XX:+EnableJVMCI"
  ]

Such a list may be needed if, for example, you need to control exactly which modules get linked in and which don't, or if your app itself isn't modularized.

chumer commented 10 months ago

Note that most of these module names are internal and are subject to change between releases. So if you can avoid it generate the list somehow from the Maven ground-truth.