Scala `2.13.14` depends on `jline-3.25.1`, which contains a classfile that is not compatible with `JDK17` (only `JDK21` is supported)

[panbingkun]

Reproduction steps

When we were preparing to upgrade the scala version from 2.13.13 to 2.13.14 in Spark, our maven plugin maven-enforcer-plugin found a problem, as follows:

Error:  Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.4.1:enforce (enforce-versions) on project spark-tools_2.13: 
Error:  Rule 3: org.codehaus.mojo.extraenforcer.dependencies.EnforceBytecodeVersion failed with message:
Error:  Found Banned Dependency: org.jline:jline:jar:3.25.1

[INFO] Restricted to JDK 17 yet org.jline:jline:jar:3.25.1:compile contains org/jline/terminal/impl/ffm/CLibrary$termios.class targeted to 65.-257

The root cause is that the class org/jline/terminal/impl/ffm/CLibrary$termios.class in jline-3.25.1.jar that scala 2.13.14 depends on is compiled based on JDK21.

javap -verbose org/jline/terminal/impl/ffm/CLibrary$termios.class

Other classes in jline-3.25.1.jar seem normal, such as:

javap -verbose org/jline/utils/NonBlockingReaderImpl.class

Scala version: 2.13.14

Problem

Explain how the above behavior isn't what you expected. So our scala 2.13.14 can onlyrun on JDK21? Or is that a issue/bug ?

For details of problems encountered in upgrading, please refer to: https://github.com/apache/spark/pull/46288

[panbingkun]

cc @SethTisue @dongjoon-hyun @LuciferYang

[som-snytt]

The jline readme explains terminal support.

https://github.com/jline/jline3

The FFM provider is available since JLine 3.24 and when running on JDK >= 21.

Three cheers for dynamic linking and virtual dispatch.

The linked maven tooling issue is out of scope.

[panbingkun]

The jline readme explains terminal support.

https://github.com/jline/jline3

The FFM provider is available since JLine 3.24 and when running on JDK >= 21.

Three cheers for dynamic linking and virtual dispatch.

The linked maven tooling issue is out of scope.

So it means that scala 2.13.14 can only run in environments above JDK >= 21, right?

[som-snytt]

I will refrain from speculation.

➜  ~ sdk use java 8.0.352-tem

Using java version 8.0.352-tem in this shell.
➜  ~ scala
Welcome to Scala 2.13.14 (OpenJDK 64-Bit Server VM, Java 1.8.0_352).
Type in expressions for evaluation. Or try :help.

scala> 42
val res0: Int = 42

scala>

[SethTisue]

it means that scala 2.13.14 can only run in environments above JDK >= 21, right?

Incorrect. The class in question is never loaded on older JDKs. Scala 2.13.14, including the REPL, works just fine on JDK 8.

It's only certain kinds of tooling that can become confused by the presence of the class in question.

See https://github.com/jline/jline3/issues/937 for full details on this.

Note that Scala 2.13.15 will use JLine 3.26.1 (or higher), which has some further changes in this area (https://github.com/jline/jline3/pull/949).

At the time we do the 3.26.1 upgrade, I'll look at whether it would work to use the new JAR with the jdk8 classifier. Regardless, I do not believe there is any actual issue here — at least, unless some further evidence is brought to light.

[panbingkun]

Incorrect. The class in question is never loaded on older JDKs. Scala 2.13.14, including the REPL, works just fine on JDK 8.

Thank you for your detailed explanation. I double check org.jline:jline:3.25.1 on local en . The process is as follows: https://github.com/apache/spark/pull/46288#issuecomment-2095480361

When the jline loads org.jline.terminal.impl.ffm.FfmTerminalProvider, it skips automatically because of the exception:

java.lang.UnsupportedClassVersionError: org/jline/terminal/impl/ffm/FfmTerminalProvider has been compiled by a more recent version of the Java Runtime (class file version 65.65535), this version of the Java Runtime only recognizes class file versions up to 61.0

which will not affect us in the end.

[SethTisue]

I'll look at whether it would work to use the new JAR with the jdk8 classifier

@sideeffffect beat me to it and we are going this route for 2.13.16: https://github.com/scala/scala/pull/10889