scala-cli doesn't work with `2.13.11-M2`

[clouds56]

Version(s) Scala CLI version: 1.0.0-RC2 Scala version (default): 3.2.2

Describe the bug scala-cli run -j system -S 2.13.11-M2 --spark-standalone test.scala shows error Error: Could not find or load main class org.apache.spark.deploy.SparkSubmit while scala-cli run -j system -S 2.13.nightly--spark-standalone test.scala works well

To Reproduce with file (in example)

//> using jvm system
//> using dep org.apache.spark:spark-sql_2.13:3.4.0
//> using javaOpt --add-opens=java.base/sun.nio.ch=ALL-UNNAMED

import org.apache.spark._
import org.apache.spark.sql._

object Job {
  def main(args: Array[String]): Unit = {
    val spark = SparkSession.builder()
      .appName("Test job")
      .getOrCreate()
    import spark.implicits._
    def sc    = spark.sparkContext
    val accum = sc.longAccumulator
    sc.parallelize(1 to 10).foreach(x => accum.add(x))
    println("Result: " + accum.value)
  }
}

and run scala-cli run -j system -S 2.13.11-M2 --spark-standalone test.scala

Expected behaviour scala-cli would resolves org.apache.spark::spark-sql:3.4.0 to org.apache.spark:spark-sql_2.13.11-M2:3.4.0, expect org.apache.spark:spark-sql_2.13:3.4.0

[SethTisue]

not sure if scala-cli has handling of -bin-, but maybe instead of 2.13.11-M2 we should have called it 2.13.11-bin-M2

the primary purpose of 2.13.11-M2 was to re-bootstrap the compiler, rather than to be for general use; though of course it ought to work regardless

consider using 2.nightly instead (which at the moment is 2.13.11-bin-366ba4a)

[clouds56]

I think M2 refers to https://github.com/scala/scala/releases/tag/v2.13.11-M2, its just like RC. Is there any document that says it is used for re-bootstrap? I've tried scala-cli run -j system -S 2.13.0-RC2 --dep a::b:1.0 test.sc it also resolves to

[error]  Error downloading a:b_2.13.0-RC2:1.0
  not found: /home/clouds/.ivy2/local/a/b_2.13.0-RC2/1.0/ivys/ivy.xml
  not found: https://repo1.maven.org/maven2/a/b_2.13.0-RC2/1.0/b_2.13.0-RC2-1.0.pom
  not found: /home/clouds/.cache/scalacli/local-repo/v1.0.0-RC2/a/b_2.13.0-RC2/1.0/ivys/ivy.xml
  No fallback URL found

consider using 2.nightly instead (which at the moment is 2.13.11-bin-366ba4a)

When using 2.nightly, it would download some file from https://scala-ci.typesafe.com/ui/api/v1/ui/nativeBrowser/scala-integration/org/scala-lang/scala-compiler every single time.

[SethTisue]

Is there any document that says it is used for re-bootstrap?

Not really, except that

• the M is a standard convention that indicates that it's just a milestone with no particular guarantees around its contents or usability
• its absence from https://github.com/scala/scala/releases, and the lack of any other announcements through other channels either, is significant
• https://contributors.scala-lang.org/t/scala-2-13-11-release-planning/6088 would be a logical place to look and/or ask
• those following the Scala 2 development process have seen https://github.com/scala/bug/issues/12783 and/or the PRs that are linked from it

When using 2.nightly, it would download some file from scala-ci.typesafe.com/ui/api/v1/ui/nativeBrowser/scala-integration/org/scala-lang/scala-compiler every single time

Indeed, but on first run it you'll see a fixed version number that you're free to swap in for the varying one. That's why I included "which at the moment is 2.13.11-bin-366ba4a"

[clouds56]

I mean we at least should handle RC besides -bin-, like 2.13.0-RC2. If additionally 2.13.11-M2 could be handled it would definitely better.

Indeed, but on first run it you'll see a fixed version number that you're free to swap in for the varying one. That's why I included "which at the moment is 2.13.11-bin-366ba4a"

I tried scala-cli run -j system -S 2.13.11-bin-366ba4a test.sc this but it would still download https://scala-ci.typesafe.com/artifactory/scala-integration/org/scala-lang/scala-library/ every time

[SethTisue]

I tried scala-cli run -j system -S 2.13.11-bin-366ba4a test.sc this but it would still download scala-ci.typesafe.com/artifactory/scala-integration/org/scala-lang/scala-library every time

I see. That might merit a separate ticket?