search

sbt / zinc

Scala incremental compiler library, used by sbt and other build tools
Apache License 2.0
335 stars 121 forks source link

package object causes "Symbol 'type X' is missing from the classpath." #690

Closed eed3si9n closed 3 years ago

eed3si9n commented 5 years ago

steps

For example running Play's 2.7.x branch with 1.3.0-RC3 on Community Builds - https://scala-ci.typesafe.com/job/scala-2.12.x-integrate-community-build/4719/console

problem

[playframework] [error] Symbol 'type play.api.libs.ws.WSBodyWritables' is missing from the classpath.
[playframework] [error] This symbol is required by 'package play.api.libs.ws.package'.
[playframework] [error] Make sure that type WSBodyWritables is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
[playframework] [error] A full rebuild may help if 'package.class' was compiled against an incompatible version of play.api.libs.ws.
[playframework] [error] one error found

expectation

Compiles. It works locally on my machine.

notes

See also https://github.com/scala/scala-dev/issues/441

@dwijnand noted

I'm getting that Play-WS errors locally with 1.2.8. So it seems to be a pre-existing incremental compiler bug (IIUC).

@retronym found a standalone reproduction:

// src/main-changes/scala/p1/O1.scala
package p1

object O1 {
  def o1 = 42
}

// src/main/scala/p1/O1.scala
package p1

object O1 {
  def o1 = ""
}

// src/main/scala/p1/T1.scala
package p1

trait T1 {
  class C {
    O1.o1
  }
}

// src/main/scala/p1/package.scala
package object p1 extends T1
./build.sh
+ sbt compile
[info] welcome to sbt 1.5.3 (AdoptOpenJDK Java 11.0.11)
[info] loading global plugins from /Users/jz/.sbt/1.0/plugins
[info] loading project definition from /Users/jz/code/zinc-bug-implicit-in-package-object-parent/project
[info] loading settings for project zinc-bug-implicit-in-package-object-parent from build.sbt ...
[info] set current project to zinc-bug-implicit-in-package-object-parent (in build file:/Users/jz/code/zinc-bug-implicit-in-package-object-parent/)
[info] Executing in batch mode. For better performance use sbt's shell
[info] compiling 3 Scala sources to /Users/jz/code/zinc-bug-implicit-in-package-object-parent/target/scala-2.12/classes ...
[success] Total time: 3 s, completed 3 Jun. 2021, 6:47:18 pm
+ cp -R src/main-changes/scala src/main/
+ sbt debug compile
[info] welcome to sbt 1.5.3 (AdoptOpenJDK Java 11.0.11)
[info] loading global plugins from /Users/jz/.sbt/1.0/plugins
[info] loading project definition from /Users/jz/code/zinc-bug-implicit-in-package-object-parent/project
[info] loading settings for project zinc-bug-implicit-in-package-object-parent from build.sbt ...
[info] set current project to zinc-bug-implicit-in-package-object-parent (in build file:/Users/jz/code/zinc-bug-implicit-in-package-object-parent/)
[info] Executing in batch mode. For better performance use sbt's shell
[debug] not up to date. inChanged = true, force = false
[debug] Updating ...
[debug] Done updating
[debug] [zinc] IncrementalCompile -----------
[debug] IncrementalCompile.incrementalCompile
[debug] previous = Stamps for: 6 products, 3 sources, 1 libraries
[debug] current source = Set(${BASE}/src/main/scala/p1/T1.scala, ${BASE}/src/main/scala/p1/O1.scala, ${BASE}/src/main/scala/p1/package.scala)
[debug] > initialChanges = InitialChanges(Changes(added = Set(), removed = Set(), changed = Set(${BASE}/src/main/scala/p1/O1.scala), unmodified = ...),Set(),Set(),API Changes: Set())
[debug]
[debug] Initial source changes:
[debug]     removed: Set()
[debug]     added: Set()
[debug]     modified: Set(${BASE}/src/main/scala/p1/O1.scala)
[debug] Invalidated products: Set()
[debug] External API changes: API Changes: Set()
[debug] Modified binary dependencies: Set()
[debug] Initial directly invalidated classes: Set(p1.O1)
[debug] Sources indirectly invalidated by:
[debug]     product: Set()
[debug]     binary dep: Set()
[debug]     external source: Set()
[debug] All initially invalidated classes: Set(p1.O1)
[debug] All initially invalidated sources:Set(${BASE}/src/main/scala/p1/O1.scala)
[debug] Created transactional ClassFileManager with tempDir = /Users/jz/code/zinc-bug-implicit-in-package-object-parent/target/scala-2.12/classes.bak
[debug] Initial set of included nodes: p1.O1
[debug] About to delete class files:
[debug]     O1.class
[debug]     O1$.class
[debug] We backup class files:
[debug]     O1.class
[debug]     O1$.class
[debug] compilation cycle 1
[info] compiling 1 Scala source to /Users/jz/code/zinc-bug-implicit-in-package-object-parent/target/scala-2.12/classes ...
[debug] Getting org.scala-sbt:compiler-bridge_2.12:1.5.4:compile for Scala 2.12.14
[debug] [zinc] Running cached compiler 64006605 for Scala compiler version 2.12.14
[debug] [zinc] The Scala compiler is invoked with:
[debug]     -bootclasspath
[debug]     /Users/jz/.sbt/boot/scala-2.12.14/lib/scala-library.jar
[debug]     -classpath
[debug]     /Users/jz/code/zinc-bug-implicit-in-package-object-parent/target/scala-2.12/classes
[debug] Invalidating (transitively) by inheritance from p1.O1...
[debug] Initial set of included nodes: p1.O1
[debug] Invalidated by transitive inheritance dependency: Set(p1.O1)
[debug] The following modified names cause invalidation of p1.T1.C: Set(UsedName(o1,[Default]))
[debug] Change NamesChange(p1.O1,ModifiedNames(changes = UsedName(o1,[Default]))) invalidates 2 classes due to The p1.O1 has the following regular definitions changed:
[debug]     UsedName(o1,[Default]).
[debug]   > by transitive inheritance: Set(p1.O1)
[debug]   >
[debug]   > by member reference: Set(p1.T1.C)
[debug]
[debug] New invalidations:
[debug]     p1.T1.C
[debug] Initial set of included nodes: p1.T1.C
[debug] Previously invalidated, but (transitively) depend on new invalidations:
[debug] Final step, transitive dependencies:
[debug]     Set(p1.T1.C)
[debug] Invalidated classes: p1.T1.C
[debug] Scala compilation took 1.966540712 s
[debug] done compiling
[debug] Registering generated classes:
[debug]     O1$.class
[debug]     O1.class
[debug] Initial set of included nodes: p1.T1.C
[debug] About to delete class files:
[debug]     T1$C.class
[debug]     T1.class
[debug] We backup class files:
[debug]     T1$C.class
[debug]     T1.class
[debug] compilation cycle 2
[info] compiling 1 Scala source to /Users/jz/code/zinc-bug-implicit-in-package-object-parent/target/scala-2.12/classes ...
[debug] Getting org.scala-sbt:compiler-bridge_2.12:1.5.4:compile for Scala 2.12.14
[debug] [zinc] Running cached compiler 5d107f56 for Scala compiler version 2.12.14
[debug] [zinc] The Scala compiler is invoked with:
[debug]     -bootclasspath
[debug]     /Users/jz/.sbt/boot/scala-2.12.14/lib/scala-library.jar
[debug]     -classpath
[debug]     /Users/jz/code/zinc-bug-implicit-in-package-object-parent/target/scala-2.12/classes
[error] Symbol 'type p1.T1' is missing from the classpath.
[error] This symbol is required by 'package p1.package'.
[error] Make sure that type T1 is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
[error] A full rebuild may help if 'package.class' was compiled against an incompatible version of p1.
[error] one error found
[debug] Compilation failed (CompilerInterface)
[debug] Rolling back changes to class files.
[debug] Removing generated classes:
[debug]     O1.class
[debug]     O1$.class
[debug] Restoring class files:
[debug]     T1.class
[debug]     T1$C.class
[debug]     O1$.class
[debug]     O1.class
[debug] Removing the temporary directory used for backing up class files: /Users/jz/code/zinc-bug-implicit-in-package-object-parent/target/scala-2.12/classes.bak
[error] (Compile / compileIncremental) Compilation failed
+ R=1
+ git co -f HEAD -- src/main/
+ exit 1
eed3si9n commented 5 years ago

package.scala

https://github.com/playframework/playframework/blob/6c358ec7f8495a59b3b9847a3649cf0321c96765/transport/client/play-ws/src/main/scala/play/api/libs/ws/package.scala

/*
 * Copyright (C) 2009-2019 Lightbend Inc. <https://www.lightbend.com>
 */

package play.api.libs

/**
 * Provides implicit type classes when you import the package.
 */
package object ws extends WSBodyReadables with WSBodyWritables

WSBodyReadables.scala

https://github.com/playframework/playframework/blob/6c358ec7f8495a59b3b9847a3649cf0321c96765/transport/client/play-ws/src/main/scala/play/api/libs/ws/WSBodyReadables.scala

/*
 * Copyright (C) 2009-2019 Lightbend Inc. <https://www.lightbend.com>
 */

package play.api.libs.ws

/**
 * JSON, XML and Multipart Form Data Readables used for Play-WS bodies.
 */
trait WSBodyReadables extends DefaultBodyReadables with JsonBodyReadables with XMLBodyReadables

object WSBodyReadables extends WSBodyReadables

WSBodyWritables.scala

https://github.com/playframework/playframework/blob/6c358ec7f8495a59b3b9847a3649cf0321c96765/transport/client/play-ws/src/main/scala/play/api/libs/ws/WSBodyWritables.scala

/*
 * Copyright (C) 2009-2019 Lightbend Inc. <https://www.lightbend.com>
 */

package play.api.libs.ws

import akka.stream.scaladsl.Source
import akka.util.ByteString
import play.api.mvc.MultipartFormData
import play.core.formatters.Multipart

/**
 * JSON, XML and Multipart Form Data Writables used for Play-WS bodies.
 */
trait WSBodyWritables extends DefaultBodyWritables with JsonBodyWritables with XMLBodyWritables {

  implicit val bodyWritableOf_Multipart: BodyWritable[Source[MultipartFormData.Part[Source[ByteString, _]], _]] = {
    val boundary    = Multipart.randomBoundary()
    val contentType = s"multipart/form-data; boundary=$boundary"
    BodyWritable(b => SourceBody(Multipart.transform(b, boundary)), contentType)
  }

}

object WSBodyWritables extends WSBodyWritables

notes

It's interesting to think about the semantics of this compilation. Let's say you start from the top.

  1. To compile package.scala you need WSBodyReadables and WSBodyWritables
  2. Next you try to compile WSBodyReadables.scala. The trait WSBodyReadables is inside play.api.libs.ws package. Does it expose any implicits? We know it does because of WSBodyReadables, but package.class doesn't exist yet.
eatkins commented 5 years ago

This may not be a zinc bug. The circe community build was fixed by https://github.com/sbt/sbt/pull/4949 on my computer.

eed3si9n commented 5 years ago

A potential variant when changing branches in scala/scala - https://gist.github.com/eed3si9n/468c902ea116893c4b86d31b205c489c

retronym commented 3 years ago

I found a reproduction and updated the description of this bug with the details. I'm looking for a fix now.