build Scala 3 using Fury

[propensive]

It should be possible to write a Fury build for building the Dotty compiler, and then to use that compiler to compile downstream projects. This may depend on Bloop's ability to accept a just-built compiler to build downstream projects, but I think Fury's model should already support the possibility.

[propensive]

/cc @milessabin

[propensive]

@smarter has given me a description of the existing Dotty build. Here is a transcript:

@smarter No, however I'm planning to refactor it and document it more my plan is to distinguish three stages, like some other compilers do stage 0 is the reference compiler (currently for Dotty that's Scala 2.12.8) stage 1 is the compiler you get by compiling the sources in the Dotty project using the stage 0 compiler stage 2 is the compiler you get by compiling the sources in the Dotty project using the stage 1 compiler and is what we distribute (currently that's the -bootstrapped projects in the dotty build) Now, the additional complication is that there's both dotty-library and dotty-compiler

@propensive Ok, I'm listening. (Thank you for this!)

@smarter and that there's both a JVM classpath which is the -classpath you pass to java, and a compiler classpath (also called "Scala classpath") which is the -classpath you pass to the compiler Main The JVM classpath matters for loading classes on the JVM The compiler classpath matters for loading symbols in the compiler To compile the stage 1 library, you need the stage 0 compiler (and therefore the stage 0 library) on the JVM classpath The stage 0 compiler also expects the stage 0 library or something compatible with it to be on the compiler classpath this is made more confusing because for Dotty, the standard library is both scala-library and dotty-library, eventually both of these things should be merged, but we can't do that as long as we want to use Scala 2 libraries in Dotty projects due to binary compatibility issues

@propensive When you say there's a classpath passed into the compiler Main, you mean that: you run java, you give it a -classpath, you specify that the main method is in the Main object, and that takes another -classpath as an argument? @smarter To compile the stage 1 compiler, we use the stage 0 compiler and library on the JVM classpath, and the stage 1 library on the compiler classpath Yes.

@propensive Got it, great.

@smarter Now here's where it gets really confusing: to compile the stage 2 library, we use the stage 1 compiler and library on the JVM classpath, and nothing on the compiler classpath, because the stage 1 compiler will just pick up symbols from the library sources it's compiling and there's an extra technical complication: because the compiler will try to load symbols from the library very early on, before it had a chance to index symbols from the sources it's currently compiling, we need to also pass the library sources using -sourcepath

@propensive -sourcepath is a parameter to Main from stage1?

@smarter yes Finally, the stage 2 compiler is compiled using the stage 1 compiler and library on the JVM classpath and the stage 2 library on the compiler classpath Don't worry if you didn't get any of that, it's all very confusing and I'm not doing a great job at explaining it :) I'd need a lot of diagrams Maybe give the most boring talk in the world about this

@propensive I'm going to copy/paste it into my open Fury issue on this. :) Does each stage involve more than two invocations of the compiler? (One for the n+1 compiler, and one for the n+1 library)

@smarter I also wrote some stuff here: https://github.com/lampepfl/dotty/blob/3c9935f5c8adcc40c8c218de45ef9b32c0fb328e/sbt-dotty/src/dotty/tools/sbtplugin/DottyPlugin.scala#L196-L230 No At the beginning we also need to compile the dotty-sbt-bridge but we rewrote that in Java to simplify bootstrapping

@propensive And is there a list of dependencies (other than the obvious ones)? When I say "obvious ones" I mean the standard library. Nothing else is obvious to me. ;)

@smarter

> show dotty-library/dependencyClasspath
[info] * Attributed(/home/smarter/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.8.jar)
> show dotty-compiler/dependencyClasspath
[info] * Attributed(/home/smarter/opt/dotty/interfaces/target/classes)
[info] * Attributed(/home/smarter/opt/dotty/library/target/scala-2.12/classes)
[info] * Attributed(/home/smarter/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.8.jar)
[info] * Attributed(/home/smarter/.ivy2/cache/org.scala-lang.modules/scala-asm/bundles/scala-asm-6.0.0-scala-1.jar)
[info] * Attributed(/home/smarter/.ivy2/cache/org.scala-lang.modules/scala-xml_2.12/bundles/scala-xml_2.12-1.1.0.jar)
[info] * Attributed(/home/smarter/.ivy2/cache/org.scala-sbt/compiler-interface/jars/compiler-interface-1.2.5.jar)
[info] * Attributed(/home/smarter/.ivy2/cache/org.scala-sbt/util-interface/jars/util-interface-1.2.2.jar)
[info] * Attributed(/home/smarter/.ivy2/cache/org.jline/jline-reader/jars/jline-reader-3.9.0.jar)
[info] * Attributed(/home/smarter/.ivy2/cache/org.jline/jline-terminal/jars/jline-terminal-3.9.0.jar)
[info] * Attributed(/home/smarter/.ivy2/cache/org.jline/jline-terminal-jna/jars/jline-terminal-jna-3.9.0.jar)
[info] * Attributed(/home/smarter/.ivy2/cache/net.java.dev.jna/jna/jars/jna-4.2.2.jar)

@propensive Wonderful. Thank you! I'll let you know how I get on...

@smarter I forgot to mention dotty-interfaces, which is just Java-defined interfaces to call the compiler without binary-compatibility issues

@propensive Oh, so that needs to be compiled with Java beforehand?

@smarter yeah