search

sbt / sbt

sbt, the interactive build tool
https://scala-sbt.org
Apache License 2.0
4.8k stars 937 forks source link

cross publishing using `+publishLocal` with subprojects ignores publish settings in sbt 1.1.1 #3992

Open ucbjrl opened 6 years ago

ucbjrl commented 6 years ago

This is distilled from a larger, more complex project, but seems to demonstrate the problem. My layout is:

top/
  project/plugins.sbt
  src/main/scala/hello/main.scala
  src/test/scala/hello/Test.scala
  world/src/main/scala/hello/sub/World.scala
  build.sbt

We want to generate coverage for the root project and all sub-projects which currently requires that we aggregate sub-projects in the root project. We want to publish a single jar containing code from the root project and all sub-projects so we want to disable aggregation on publish for the sub-projects and use mappings to add sub-project mappings to the root project.

As a result, our build.sbt is a little complex. After much trial and error, we have come up with the following which does the job:

name := "projwithsub"

version := "0.1-SNAPSHOT"

lazy val commonSettings = Seq(
  scalaVersion := "2.12.4",
  crossScalaVersions := Seq("2.11.12", "2.12.4")
)

lazy val world = (project in file("world"))
  .settings(commonSettings: _*)
  .settings(
    publishArtifact := false,
    publishTo := Some(Resolver.file("Unused transient repository", file("target/unusedrepo"))),
  )

lazy val hello = (project in file("."))
  .dependsOn(world)
  .aggregate(world)
  .settings(commonSettings: _*)
  .settings(
    aggregate := false,
    aggregate in coverageReport := true,
    mappings in (Compile, packageBin) ++= (mappings in (world, Compile, packageBin)).value,
    mappings in (Compile, packageSrc) ++= (mappings in (world, Compile, packageSrc)).value,
    libraryDependencies ++= Seq(
      "org.scalatest" %% "scalatest" % "3.0.1" % "test",
      "org.scalacheck" %% "scalacheck" % "1.13.4" % "test",
    )
  )

steps

% sbt clean coverage test
% sbt coverageReport
% sbt coverageAggregate

generates the aggregated coverage report for the root project and all sub-projects, and

% sbt publishLocal

publishes a single jar containing code from the root-project and all sub-projects.

problem

% sbt +clean +compile +publishLocal

seems to ignore the attempts to turn off aggregation for the publish task and publishes both the root project jar and the sub-project jars.

Additionally, after removing the target directories and any previously published $HOME/.ivy2/local/world directory:

% sbt +clean +publishLocal

results in:

cd .../top/
sbt +clean +publishLocal
[info] Loading settings from idea.sbt,plugins.sbt,sbt-pgp.sbt ...
[info] Loading global plugins from $HOME/.sbt/1.0/plugins
[info] Loading settings from plugins.sbt ...
[info] Loading project definition from .../top/project
[info] Updating ProjectRef(uri("file:.../top/project/"), "top-build")...
[info] Done updating.
[info] Loading settings from build.sbt ...
[info] Set current project to projwithsub (in build file:.../top/)
[info] Setting Scala version to 2.12.4 on 2 projects.
[info] Reapplying settings...
[info] Set current project to projwithsub (in build file:.../top/)
[success] Total time: 0 s, completed Mar 6, 2018 11:30:01 AM
[info] Setting Scala version to 2.11.12 on 2 projects.
[info] Reapplying settings...
[info] Set current project to projwithsub (in build file:.../top/)
[success] Total time: 0 s, completed Mar 6, 2018 11:30:02 AM
[info] Reapplying settings...
[info] Set current project to projwithsub (in build file:.../top/)
[info] Setting Scala version to 2.12.4 on 2 projects.
[info] Reapplying settings...
[info] Set current project to projwithsub (in build file:.../top/)
[info] Updating world...
[info] :: delivering :: world#world_2.12;0.1.0-SNAPSHOT :: 0.1.0-SNAPSHOT :: integration :: Tue Mar 06 11:30:03 PST 2018
[info] Packaging .../top/target/scala-2.12/projwithsub_2.12-0.1-SNAPSHOT-sources.jar ...
[info] Done packaging.
[info] Done updating.
[info] Updating ...
[info] Compiling 1 Scala source to .../top/world/target/scala-2.12/classes ...
[info] Wrote .../top/target/scala-2.12/projwithsub_2.12-0.1-SNAPSHOT.pom
[info] Done updating.
[info] Done compiling.
[info] Compiling 1 Scala source to .../top/target/scala-2.12/classes ...
[info] Main Scala API documentation to .../top/target/scala-2.12/api...
model contains 3 documentable templates
[info] Done compiling.
[info] Packaging .../top/target/scala-2.12/projwithsub_2.12-0.1-SNAPSHOT.jar ...
[info] Done packaging.
[info] Main Scala API documentation successful.
[info] Packaging .../top/target/scala-2.12/projwithsub_2.12-0.1-SNAPSHOT-javadoc.jar ...
[info] Done packaging.
[info] :: delivering :: projwithsub#projwithsub_2.12;0.1-SNAPSHOT :: 0.1-SNAPSHOT :: integration :: Tue Mar 06 11:30:05 PST 2018
[info]  delivering ivy file to .../top/target/scala-2.12/ivy-0.1-SNAPSHOT.xml
[info]  published projwithsub_2.12 to $HOME/.ivy2/local/projwithsub/projwithsub_2.12/0.1-SNAPSHOT/poms/projwithsub_2.12.pom
[info]  published projwithsub_2.12 to $HOME/.ivy2/local/projwithsub/projwithsub_2.12/0.1-SNAPSHOT/jars/projwithsub_2.12.jar
[info]  published projwithsub_2.12 to $HOME/.ivy2/local/projwithsub/projwithsub_2.12/0.1-SNAPSHOT/srcs/projwithsub_2.12-sources.jar
[info]  published projwithsub_2.12 to $HOME/.ivy2/local/projwithsub/projwithsub_2.12/0.1-SNAPSHOT/docs/projwithsub_2.12-javadoc.jar
[info]  published ivy to $HOME/.ivy2/local/projwithsub/projwithsub_2.12/0.1-SNAPSHOT/ivys/ivy.xml
[error] java.lang.IllegalStateException: Ivy file not found in cache for world#world_2.12;0.1.0-SNAPSHOT!
[error]     at sbt.internal.librarymanagement.ResolutionCache.getResolvedModuleDescriptor(ResolutionCache.scala:66)
[error]     at org.apache.ivy.core.deliver.DeliverEngine.deliver(DeliverEngine.java:111)
[error]     at org.apache.ivy.Ivy.deliver(Ivy.java:601)
[error]     at sbt.internal.librarymanagement.IvyActions$.$anonfun$deliver$2(IvyActions.scala:97)
[error]     at sbt.internal.librarymanagement.IvySbt$Module.$anonfun$withModule$1(Ivy.scala:243)
[error]     at sbt.internal.librarymanagement.IvySbt.$anonfun$withIvy$1(Ivy.scala:204)
[error]     at sbt.internal.librarymanagement.IvySbt.sbt$internal$librarymanagement$IvySbt$$action$1(Ivy.scala:70)
[error]     at sbt.internal.librarymanagement.IvySbt$$anon$3.call(Ivy.scala:77)
[error]     at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:95)
[error]     at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:80)
[error]     at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:99)
[error]     at xsbt.boot.Using$.withResource(Using.scala:10)
[error]     at xsbt.boot.Using$.apply(Using.scala:9)
[error]     at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:60)
[error]     at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
[error]     at xsbt.boot.Locks$.apply0(Locks.scala:31)
[error]     at xsbt.boot.Locks$.apply(Locks.scala:28)
[error]     at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.scala:77)
[error]     at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:199)
[error]     at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:196)
[error]     at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.scala:242)
[error]     at sbt.internal.librarymanagement.IvyActions$.deliver(IvyActions.scala:92)
[error]     at sbt.internal.librarymanagement.IvyActions$.publish(IvyActions.scala:126)
[error]     at sbt.Classpaths$.$anonfun$publishTask$4(Defaults.scala:2240)
[error]     at sbt.Classpaths$.$anonfun$publishTask$4$adapted(Defaults.scala:2239)
[error]     at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error]     at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:39)
[error]     at sbt.std.Transform$$anon$4.work(System.scala:66)
[error]     at sbt.Execute.$anonfun$submit$2(Execute.scala:262)
[error]     at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error]     at sbt.Execute.work(Execute.scala:271)
[error]     at sbt.Execute.$anonfun$submit$1(Execute.scala:262)
[error]     at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:174)
[error]     at sbt.CompletionService$$anon$2.call(CompletionService.scala:36)
[error]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[error]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[error]     at java.lang.Thread.run(Thread.java:745)
[error] (world / publishLocal) java.lang.IllegalStateException: Ivy file not found in cache for world#world_2.12;0.1.0-SNAPSHOT!
[error] Total time: 2 s, completed Mar 6, 2018 11:30:05 AM

Compilation exited abnormally with code 1 at Tue Mar  6 11:30:05

whereas both:

% sbt +clean publishLocal

or

% sbt +clean ++2.11.12 publishLocal

behave as expected.

expectation

+publishLocal (and +publishLocalSigned and +publishSigned) should honor the same publish settings as publishLocal (and publishLocalSigned and publishSigned respectively)

notes

project/plugins.sbt:

addSbtPlugin("org.scoverage" % "sbt-scoverage" % "1.6.0-M3")

src/main/scala/hello/main.scala:

package hello

import hello._

object Hello {
  def main(args: Array[String]) {
    println("Hello, " + sub.World.string)
  }
}

src/test/scala/hello/Test.scala:

package hello

import org.scalatest._

class Test extends FlatSpec {
  "basic code" should "work" in {
    Hello.main(Array())
  }
}

world/src/main/scala/hello/sub/World.scala:

package hello.sub

object World {
  val string = "world"
  println("world ready")
}

sbt version: 1.1.1

arbitrary-dev commented 6 years ago

I have a similar problem, but when placing publishArtifact := false on a root project. Problem somehow disappears as I put this line on top of build.sbt:

version := "1.0"
eed3si9n commented 6 years ago

@ucbjrl Thanks for the report. I guess this is one reason to move towards more immutable cross building. See https://github.com/sbt/sbt-projectmatrix