Library dependency missing in `managedClasspath` when upgrading from sbt 0.13.16-M1 to 0.13.16-RC1

dwijnand commented 7 years ago

steps

Set sbt.version to 0.13.16-RC1 (or 0.13.16) and use the following build:

val a = project settings (
         organization := "org.scala-sbt",
                 name := "Ivy",
           crossPaths := false,
  libraryDependencies += "junit" % "junit" % "4.11" % Test
)

val b = project settings (
  organization := "org.scala-sbt",
          name := "sbt",
       version := "0.13.16-SNAPSHOT",
    crossPaths := false
)

val z = project dependsOn (a % "test->test", b) settings (sbtPlugin := true)

problem

Invoke show z/test:managedClasspath:

[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/sbt/jars/sbt-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.sbt/boot/scala-2.10.6/lib/scala-library.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/main/jars/main-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/actions/jars/actions-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/classpath/jars/classpath-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.sbt/boot/scala-2.10.6/lib/scala-compiler.jar)
[info] * Attributed(/Users/dnw/.sbt/boot/scala-2.10.6/lib/scala-reflect.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/interface/jars/interface-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/io/jars/io-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/control/jars/control-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/launcher-interface/jars/launcher-interface-1.0.1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/completion/jars/completion-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/collections/jars/collections-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/jline/jline/jars/jline-2.14.4.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/api/jars/api-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/classfile/jars/classfile-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/logging/jars/logging-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/process/jars/process-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/compiler-integration/jars/compiler-integration-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/incremental-compiler/jars/incremental-compiler-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/relation/jars/relation-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/compile/jars/compile-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/persist/jars/persist-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-tools.sbinary/sbinary_2.10/jars/sbinary_2.10-0.4.2.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/compiler-ivy-integration/jars/compiler-ivy-integration-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/ivy/jars/ivy-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/cross/jars/cross-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt.ivy/ivy/jars/ivy-2.3.0-sbt-48dd0744422128446aee9ac31aa356ee203cc9f4.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/com.jcraft/jsch/jars/jsch-0.1.50.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/serialization_2.10/jars/serialization_2.10-0.1.2.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-lang.modules/scala-pickling_2.10/jars/scala-pickling_2.10-0.10.1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scalamacros/quasiquotes_2.10/jars/quasiquotes_2.10-2.0.1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.json4s/json4s-core_2.10/jars/json4s-core_2.10-3.2.10.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.json4s/json4s-ast_2.10/jars/json4s-ast_2.10-3.2.10.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/com.thoughtworks.paranamer/paranamer/jars/paranamer-2.6.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.spire-math/jawn-parser_2.10/jars/jawn-parser_2.10-0.6.0.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.spire-math/json4s-support_2.10/jars/json4s-support_2.10-0.6.0.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/run/jars/run-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/task-system/jars/task-system-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/tasks/jars/tasks-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/tracking/jars/tracking-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/cache/jars/cache-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/testing/jars/testing-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/test-agent/jars/test-agent-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/test-interface/jars/test-interface-1.0.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/main-settings/jars/main-settings-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/apply-macro/jars/apply-macro-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/command/jars/command-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/template-resolver/jars/template-resolver-0.1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/logic/jars/logic-0.13.16-RC1.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.scala-sbt/compiler-interface/jars/compiler-interface-0.13.16-RC1.jar)

expectation

I expect JUnit to be present in the project z's test configuration managed classpath.

notes

This is the z/test:managedClasspath using sbt 0.13.16-M1:

[info] * Attributed(/Users/dnw/.sbt/boot/scala-2.10.6/lib/scala-library.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/junit/junit/jars/junit-4.11.jar)
[info] * Attributed(/Users/dnw/.ivy2/cache/org.hamcrest/hamcrest-core/jars/hamcrest-core-1.3.jar)

The original bug is in upgrading sbt/sbt's 0.13 branch build from 0.13.16-M1 to 0.13.16 in #3408.

dwijnand commented 7 years ago

Here's where I give up: Ivy seems to behave differently between 0.13.16-M1 and 0.13.16-RC1 because "rc" has "special meaning" in Ivy, while the "m" for milestone does not.

https://github.com/sbt/ivy/blob/48dd0744422128446aee9ac31aa356ee203cc9f4/src/java/org/apache/ivy/plugins/latest/LatestRevisionStrategy.java#L159

The result of that is that:

org.scala-sbt:sbt:0.13.16-M1 is evicted over org.scala-sbt:sbt:0.13.16-SNAPSHOT, but instead
org.scala-sbt:sbt:0.13.16-SNAPSHOT is evicted over org.scala-sbt:sbt:0.13.16-RC1

because "latest-revision" ✌️😒✌️

So the sbt development lesson of the day for today is: beware that when upgrading sbt's build to or from milestone releases that eviction results might be different.

I'll fix https://github.com/sbt/sbt/pull/3408 now.

dwijnand commented 7 years ago

Before I throw away the code, here's the check task I wrote to check:

TaskKey[Unit]("check") := {
  val cp = (managedClasspath in Test in z).value
  assert(
    cp exists (_.toString contains "junit"),
    s"""
       |z/test:managedClasspath doesn't contain junit, it contains:
       |${classpathToString(cp)}
     """.trim.stripMargin
  )
}

def classpathToString(cp: Def.Classpath) =
  cp map (a => s"${a.metadata(Keys.moduleID.key)}: ${a.data}") mkString "\n"

sbt / sbt