search

scala / scala-dev

Scala 2 team issues. Not for user-facing bugs or directly actionable user-facing improvements. For build/test/infra and for longer-term planning and idea tracking. Our bug tracker is at https://github.com/scala/bug/issues
Apache License 2.0
130 stars 14 forks source link

Support features of JDK 11+ #139

Open retronym opened 8 years ago

retronym commented 8 years ago

In general, Scala now works just fine on JDK 11

For the current support status of each JDK version, see https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html

JDK9 JEP list

retronym commented 8 years ago

To whet the collective appetite, here's my branch that adapts to JEP 220 up and running:

⚡ qscala
Welcome to Scala 2.12.0-20160425-151134-5e07d57 (Java HotSpot(TM) 64-Bit Server VM, Java 9-ea).
Type in expressions for evaluation. Or try :help.

scala> java.util.List.of(1, 2, 3)
res0: java.util.List[Int] = [1, 2, 3]
varming commented 8 years ago

With less than nine months to the release of JDK9 not being able to run Scala programs on the early access releases is a serious concern, so it is great to see that work has started.

SethTisue commented 7 years ago

@retronym did 2.12.1 check some more of these boxes?

retronym commented 7 years ago

I have submitted a PR to SBT to start the ball rolling for Java 9 support. Because SBT internally uses an old version of scalac to compile build definitions, I decided to create a tool to export a jrt:/modules/*/** to rt.jar to bridge JDK9 to the older compiler.

This approach also enables your project to be compiled with, say, Scala 2.11.8, which might help some more folks test out JDK9, without first needing to upgrade to the 2.12 series.

Warning: we are yet to implement the additional access checks required by JEP-261. Until then, you can get LinkageError-s at runtime for code that passes the Scala type checker. So while I encourage testing and experimentation, use this at your own risk in production.

retronym commented 7 years ago

Discussion on the Java Language Spec changes related to modules, thread started by Eclipse JDT seeking to implement the spec. http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012433.html

ijuma commented 7 years ago

Another Java 9 issue: https://github.com/scala/bug/issues/10418

SethTisue commented 7 years ago

I've created https://github.com/scala/bug/labels/jdk9

ijuma commented 7 years ago

JEP 247: Compile for Older Platform Versions is also relevant. It would be nice if scalac could understand $JDK_ROOT/lib/ct.sym so that a newer JDK could be safely used to build for an older version.

asarkar commented 7 years ago

Is there a target release for addressing all the issues listed here, and reported by the users, or is it going to be on best-efforts basis?

retronym commented 7 years ago

@asarkar My primary goal at the moment is removing impediments to building and running Scala 2.12 on JDK9. We have been able to build and test the Play Framework, and the Akka team is working through similar testing at the moment.

I then want to implement support in the Scala typechecker for JPMS. I expect this will be done in Q4 this year. I'm hopeful that we could ship this in either a minor release of Scala 2.12, or as an experimental compiler plugin for Scala 2.12. It will introduce a dependency on the Java 9 standard library, which will complicate our build somewhat, as in general the compiler should only require Java 8.

I don't have a timeframe for the full list, but I'm keen to hear about what parts are most important for our users to help prioritize.

smarter commented 7 years ago

Update Zinc to understand that jrt:// classpath entries should have a timestamp based on the JDK (maybe $JAVA_HOME/jmods/*.mod? Find out exactly what file backs jrt://)

Surely that's not right for modules on the modulepath that are not part of the JDK?

retronym commented 7 years ago

Once we have a --modulepath compiler option, we'll be able to access the contents are regular JARs (well, apart from the .jmod format). So we might not actually access them via the jrt:// filesystem at all.

lukaseder commented 7 years ago

I managed to compile stuff on the JDK 9 using Scala 2.12.3, but not using 2.11.11 where I'm getting this error:

error: scala.reflect.internal.MissingRequirementError: object java.lang.Object in compiler mirror not found.

Is JDK 9 usage not supported with 2.11, or might there be a bug worth reporting?

lrytz commented 7 years ago

This is probably https://github.com/scala/scala-dev/issues/304, fixed for 2.12 in https://github.com/scala/scala/pull/6097. We're backporting some fixes for a 2.11.12 release, should be out soon.

lukaseder commented 7 years ago

Cook, thanks very much for the pointer, @lrytz !

mkurz commented 7 years ago

I opened pull request https://github.com/scala/scala/pull/6146 to add support for -target:jvm-1.9.

clhodapp commented 7 years ago

It would be great if Scala Team could publish an official position on the readiness of Scala for JDK/JRE9 (even if it's just a line or two integrated into the download directions on scala-lang.org). As it stands, it's not clear at all to most developers whether it's considered safe to use Scala with Java 9. This is, of course, completely separate from fully leveraging Java 9, which is obviously going to be a substantial ongoing effort for quite a while (as reflected here).

SethTisue commented 7 years ago

re: download directions on scala-lang.org, see https://github.com/scala/scala-lang/issues/761

markehammons commented 7 years ago

I would like to say

javac allows programmers to declare module requirements, exports inline in the sourcetree ("module-info.java" files). What should we do?

is of great importance to me. While I've been running my scala code on java 9 recently (and also openJ9), I haven't bothered messing with modules yet because scala doesn't support them. This is a shame imo, cause while stuff like private[package] is nice, I'd really love to have jpms modules instead.

retronym commented 7 years ago

@markehammons The plan is that you will define your module in the same way as you would in Java by adding a module-info.java file to your source tree. All the build tools support mixed Java/Scala projects.

The missing piece of the puzzle is getting scalac to respect these limitations.

I've just finished backporting the of Java 9 support we have so far from 2.12.4 back to the imminent 2.10.7 and 2.11.12 to help folks migrate. In particular, this is designed to help people using SBT 0.13 (which implies use of Scala 2.10.x).

Once those releases are done, I'll add support in 2.12.5 for respecting multi-release JARs and the --release compiler option (to limit compilation to, e.g. the Java 8 subset of the Java standard library). These are relatively low hanging fruit.

Next cab off the rank will be full support for JPMS modules.

@clhodapp After the backport releases I'll blog about the state of JDK 9 support and the roadmap.

ijuma commented 7 years ago

Happy to hear about --release. As you say, it's low hanging fruit and it simplifies the contributor experience (no need to ask them to install an older JDK).

schmitch commented 7 years ago

@retronym so the way forward to use the module-system is to place a module-info.java into the scala/ folder? doesn't that seems wierd. Can't the scala compiler have his own module-info.scala, I don't think it would be great to put *.java files in 100% scala libraries?!

ktoso commented 7 years ago

If it would be a *.scala file then one would expect not having to put ; etc, so we'd have to have our own parser perhaps for it, making is slower to adopt it (and more potential for diverging from the original). I personally don't mind the module-info.java files -- it's more of a "the java runtime cares about this" anyway and since it's a form of DSL anyway. One could add them to java/ usually I think anyway.

schmitch commented 7 years ago

well but wouldn't that mean that scala-js/scala-native needs to ingore any module-info.java?

ktoso commented 7 years ago

Indeed, since it's a java runtime thing. What about it?

saosebastiao commented 7 years ago

javac allows programmers to declare module requirements, exports inline in the sourcetree ("module-info.java" files). What should we do?

A suggestion: modify or make available several different App-like trait mixins which provide implicit values representing module capabilities. Then you statically control availability of the resulting classes and methods via implicit views or generalized type constraints. For example: 

object Main extends JVM9ModuleApp with ConsoleModule

would make System.out.* available via a dependency on the relevant jdk module, and it would expose methods like println via a (implicit console: C =:= Console) type constraint. If ConsoleModule is not mixed in, the implicit resolution would fail and println would generate a compile time error.

Just as an FYI, this approach would be considered a form of a coeffect system. Coeffects are the dual of effects: where an effect tracks what a program does to the world, a coeffect tracks what is required of it. It could also be extremely extensible...if the implicit coeffect provided represents availability of a type of code generator in the compiler, it could dispatch that code to that code generator. This would allow true unification of the available backends (scala-native, scalajs, scalajvm) into a single compiler, and allow cross compiled apps as well as typesafe client-server apps within the same compilation project. It could also enable non-jvm targets to finely control code stdlib availability as well as code generation, taking into consideration the os/platform APIs that are available. Some possibilities include:

object Client extends JsApp with ES6Gen { ... }
object Server extends JVM8App with CudaGen { ... }`

or even

object MyAndroidApp extends AndroidApp with API26 with NetworkPermissions with LocationPermissions { ... }
object MyIOSApp extends NativeApp with MulticoreRuntime with CocoaTouch { ... }

Tomas Petricek of the F# community has done a lot of work formalizing the concept of coeffects and has created an amazing website dedicated to explaining the topic. The possibilities are enough to make me giddy. I love scala as a language, and I love the direction that the community is going with it (scalajs, scala-native, etc.), but all of that unified via a coeffect system could be transformative for cross-platform programming in the same way that Rust is transforming systems programming. And the jdk9 module system is the perfect start point for it.

devshorts commented 6 years ago

Out of curiosity is there still activity on this? I realize this is a tricky ticket, but JDK10 is scheduled for GA in march http://openjdk.java.net/projects/jdk/10/ so it'd be nice to not be 2 major versions behind in scala land

smarter commented 6 years ago

@devshorts This ticket covers a wide range of Java 9 related things. It'd be helpful if you could indicate which ones in particular you're interested in. Note that the latest releases of sbt, Scala 2.10, 2.11 and 2.12 work correctly on Java 9 and produce code that can be run on Java 9.

SethTisue commented 6 years ago

@devshorts JDK 9 in general is still very much on the collective mind of the Scala and tooling teams at Lightbend and remains near the top of our priorities. (and thanks @smarter for noting some of the progress made in the last few months)

SethTisue commented 6 years ago

note that community participation in making progress on remaining checkboxes and helping them happen faster is very much welcome

smarter commented 6 years ago

Ideally, we would have a documentation page "Scala on Java 9" to point people to. The current situation is very confusing since if you have an outdated sbt/scalac you just get obscure error messages.

SethTisue commented 6 years ago

related: https://github.com/scala/scala-lang/issues/761

SethTisue commented 6 years ago

Ideally, we would have a documentation page "Scala on Java 9" to point people to

totally agree. I opened https://github.com/scala/docs.scala-lang/issues/1000 on it

xuwei-k commented 6 years ago

FYI. I have filed JDK10 issue https://github.com/scala/bug/issues/10717

SethTisue commented 6 years ago

a basic page http://docs.scala-lang.org/overviews/jdk-compatibility/overview.html is now in place, edits/expansions welcome

psr commented 6 years ago

It appears that updates to JDK 8 are going to require a license to use it in business or commercial settings beyond January 2019. The lack of support for JDK 9+ is getting more pressing!

smarter commented 6 years ago

@psr JDK 9+ is already supported by recent scalac and sbt releases, see https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html or is there a specific feature of JDK 9+ you're looking for that is missing in scalac?

psr commented 6 years ago

@smarter the advice on that page still recommends using JDK 8 for compiling Scala code. Is that now no longer the case?

smarter commented 6 years ago

It's recommended because it's the version that has received the most testing, but newer versions are supported. (There's a bug with macros and JDK 9+ with 2.12.5 but that will be fixed in 2.12.6: https://github.com/scala/scala-dev/issues/480)

psr commented 6 years ago

Ah, that's great news.

retronym commented 6 years ago

I've split out a ticket for JPMS support which I'm working on at the moment.

Teaser:

% qscalac  -nobootcp -modulepath ""  $(f "class C { def foo = javax.xml.bind.DatatypeConverter.printBase64Binary _ }")
/tmp/a.scala:1: error: object DatatypeConverter is not a member of package javax.xml.bind
class C { def foo = javax.xml.bind.DatatypeConverter.printBase64Binary _ }
                                   ^
one error found

% qscalac  -nobootcp -modulepath "" -addmodules:java.xml.bind  $(f "class C { def foo = javax.xml.bind.DatatypeConverter.printBase64Binary _ }")

% qscalac  -nobootcp -modulepath "" $(f "class C { sun.reflect.misc.ConstructorUtil.getConstructors(null) }")
/tmp/a.scala:1: error: class ConstructorUtil in package misc cannot be accessed in package sun.reflect.misc
class C { sun.reflect.misc.ConstructorUtil.getConstructors(null) }
                           ^
one error found

% qscalac  -nobootcp -modulepath "" -addexports:java.base/sun.reflect.misc=ALL-UNNAMED  $(f "class C { sun.reflect.misc.ConstructorUtil.getConstructors(null) }")
denisrosset commented 6 years ago

I've got compilation problems on Java 10 for https://github.com/denisrosset/symdpoly on 2.12.6.

(java.lang.AssertionError: assertion failed: ClassBType.info not yet assigned: Ljava/lang/Object) Should I file a bug report, or is Java 10 support only preliminary?

smarter commented 6 years ago

Open a bug report.

ijuma commented 6 years ago

The initial RC for Java 11 is due in 1 week. Time to file a ticket for Java 11 support? :)

smarter commented 6 years ago

Java 11 should work out of the box with a recent Scala and sbt, or is there a specific feature of the JDK you think scalac needs to implement?

ktoso commented 6 years ago

We have seen an Scala issue in testing Akka against JDK11: https://github.com/akka/akka/issues/25449 We are not entirely sure about it's root cause yet, please see there for details.

smarter commented 6 years ago

Please open a separate issue for this

ktoso commented 6 years ago

Right sorry, you can delete my comments and we open a separate one

ijuma commented 6 years ago

Java 11 should work out of the box with a recent Scala and sbt, or is there a specific feature of the JDK you think scalac needs to implement?

Aside from a method removal in Unsafe, the main new thing is http://openjdk.java.net/jeps/181. It currently requires ASM7_EXPERIMENTAL (https://github.com/scala/scala-asm/blob/s-6.2/src/main/java/scala/tools/asm/Opcodes.java#L55) to be used.

xuwei-k commented 6 years ago

the main new thing is http://openjdk.java.net/jeps/181.

test/files/run/t7455/Test.scala fail on JDK11. (success on JDK10)

https://github.com/scala/scala-dev/issues/559#issuecomment-425125666