Premature initialization of SVM encodings

[sgammon]

Describe the issue

When building a Truffle-enabled native-image with a certain combination of Feature implementations on the classpath, initialization of registered encodings for TruffleString may occur before the full suite of Truffle languages has loaded; this causes an invalid state which causes TruffleRuby to crash later, at runtime.

Truffle decides whether to enable JCodings based on a simple condition:

https://github.com/oracle/graal/blob/407ca89d90b9a37f0aded86e334c2711d16b411e/truffle/src/com.oracle.truffle.api.strings/src/com/oracle/truffle/api/strings/JCodings.java#L60

TStringAccessor delegates to the engine:

https://github.com/oracle/graal/blob/407ca89d90b9a37f0aded86e334c2711d16b411e/truffle/src/com.oracle.truffle.api.strings/src/com/oracle/truffle/api/strings/TStringAccessor.java#L66-L68

Which delegates to EngineAccessor:

https://github.com/oracle/graal/blob/407ca89d90b9a37f0aded86e334c2711d16b411e/truffle/src/com.oracle.truffle.polyglot/src/com/oracle/truffle/polyglot/EngineAccessor.java#L1982-L1985

Which ultimately calls into the LanguageCache:

https://github.com/oracle/graal/blob/407ca89d90b9a37f0aded86e334c2711d16b411e/truffle/src/com.oracle.truffle.polyglot/src/com/oracle/truffle/polyglot/LanguageCache.java#L214-L220

... which then queries the registered Truffle languages to determine if all codings are needed.

However, at this stage of execution, it is not guaranteed that Truffle languages have loaded yet via TruffleBaseFeature:

https://github.com/oracle/graal/blob/407ca89d90b9a37f0aded86e334c2711d16b411e/substratevm/src/com.oracle.svm.truffle/src/com/oracle/svm/truffle/TruffleBaseFeature.java#L247-L255

TruffleRuby at latest release happens to depend on all encodings. As a result, the native-image created under these conditions either breaks immediately at runtime, with:

Caused by: java.lang.IllegalArgumentException: unknown encoding: "ASCII-8BIT"
        at org.graalvm.truffle/com.oracle.truffle.api.strings.InternalErrors.illegalArgument(InternalErrors.java:76)
        at org.graalvm.truffle/com.oracle.truffle.api.strings.InternalErrors.unknownEncoding(InternalErrors.java:96)
        at org.graalvm.truffle/com.oracle.truffle.api.strings.TruffleString$Encoding.fromJCodingName(TruffleString.java:1094)
        at org.truffleruby.core.encoding.TStringUtils.jcodingToTEncoding(TStringUtils.java:43)
        at org.truffleruby.core.encoding.RubyEncoding.<init>(RubyEncoding.java:57)
        at org.truffleruby.core.encoding.Encodings.initializeRubyEncodings(Encodings.java:95)
        at org.truffleruby.core.encoding.Encodings.<clinit>(Encodings.java:43)

... or at build time with the same exception, if --initialize-at-build-time=org.truffleruby.core.encoding.Encodings is passed.

Steps to reproduce the issue

In our case (Elide), we are building against latest (24.0.1) native-image, with GraalPython, TruffleRuby, and GraalJs.

GraalPython triggers the early initialization of TruffleString encodings via a handful of calls in the BouncyCastleFeature and JNIFeature classes. In both cases, there are checks against PythonOptions static fields; for example here. It is the class initialization of PythonOptions which initializes encodings with TruffleString literals used at class-init time.

1. git clone git@github.com:elide-dev/elide.git -b main
2. cd elide && ./gradlew build -x test -x check
3. vim ./packages/cli/build.gradle.kts
4. Change enableRuby to true
5. ./gradlew :packages:cli:nativeCompile
6. ./packages/cli/build/native/nativeCompile/elide.debug shell --language=RUBY

Describe GraalVM and your environment:

• GraalVM version (latest snapshot builds can be found here), or commit id if built from source: 24.0.1
• JDK major version: 22.0.1
• OS: macOS Sonoma 14.3.1
• Architecture: M-series (aarch64)

[fernando-valdez]

Thanks for reporting this. I will take a look at it

[eregon]

This specific code changed on master, compare 24.0: https://github.com/oracle/graal/blob/release/graal-vm/24.0/truffle/src/com.oracle.truffle.api.strings/src/com/oracle/truffle/api/strings/TruffleString.java#L1049-L1051 master: https://github.com/oracle/graal/blob/c0959eeb526e2d7b4c8d12b0dd51c05c7ba0e9e1/truffle/src/com.oracle.truffle.api.strings/src/com/oracle/truffle/api/strings/JCodings.java#L143

Could you try with master too? There are dev builds at https://github.com/graalvm/graalvm-ce-dev-builds and https://github.com/graalvm/oracle-graalvm-ea-builds/releases. I suspect no error or a different error on master. The logic to detect if all encoding are needed also changed, which might have fixed this. cc @woess