scala / bug

Scala 2 bug reports only. Please, no questions — proper bug reports only.
https://scala-lang.org
230 stars 21 forks source link

Regression in scaladoc compilation when using Scala 3 definitions #12955

Closed WojciechMazur closed 3 months ago

WojciechMazur commented 3 months ago

Reproduction steps

Scala version: 2.13.13 Tested with Scala 3 versions for cross compiled project: 3.3.1 and 3.3.2

reproduction-sbt.zip run with sbt "base/publishLocal; projectC/publishLocal"

  1. Define ADT in Scala 3, create a jar out of it
    
    package testlib

enum ADT: case SingletonCase case ClassCase(x: String)


2. Define Scala 2 project that uses Scala 3 JAR 
```scala
package app

object Main {
  def main(args: Array[String]): Unit = {
    println(testlib.ADT.SingletonCase)
    println(testlib.ADT.ClassCase("foo"))
  }
}
  1. Try to create documentation for Scala 2 project, (sbt Compile/doc)

Problem

Compilation fails with

error] stack trace is suppressed; run last projectC / Compile / doc for the full output
[error] (projectC / Compile / doc) scala.reflect.internal.Types$TypeError: Unsupported Scala 3 generic tuple type scala.Tuple in bounds of type MirroredElemTypes; found in  scala.deriving.Mirror.<refinement>.
[error] Total time: 0 s, completed Feb 22, 2024, 11:42:57 AM
lrytz commented 3 months ago

cc @bishabosha

bishabosha commented 3 months ago

writing here to confirm that it works in 2.13.12, so definitely something changed

lrytz commented 3 months ago

It's possible that the bug / behavior in the tasty reader is older but more symbols are forced in the new version of scaladoc.

bishabosha commented 3 months ago

I build scala 2.13 at https://github.com/scala/scala/commit/ffc0297072ca8248daac279781d0dea5680c3a86 (the commit before any of my changes) and it works, then at the classpath changes in https://github.com/scala/scala/commit/e6232614795da850e2c834316caa2d16b83a07ed it breaks, to confirm, I build at https://github.com/scala/scala/commit/6b2c4aedc7d4d7d6a629b2ade6a6c04fe864ee14 (the commit before) and it works again.

I'm a bit surprised at this behavior, maybe there was some laziness that is now lost?

lrytz commented 3 months ago

Is it possible that before we didn't find all tasty files? Did this work for objects?

I guess I would add a few logs of when the classfile parser / tasty reader is invoked in both before and after.

bishabosha commented 3 months ago

Is it possible that before we didn't find all tasty files? Did this work for objects?

im not sure this is the reason, i.e. the tasty attribute was always in foo.class for object foo, never foo$.class, so in theory the same files should be forced.

I guess I would add a few logs of when the classfile parser / tasty reader is invoked in both before and after.

ill try that next

bishabosha commented 3 months ago

first thing I notice is that the symbol IDs are different

bishabosha commented 3 months ago

First noticed a bug in the filename logging, but not a big deal

Screenshot 2024-02-23 at 15 02 18

however what this signals is that previously Mirror was being unpickled from the classfile, resolving the sibling tasty, but now it resolved directly from the tasty file, not sure if this is some sign

bishabosha commented 3 months ago

ok, for some reason, (found by adding a debug statement here) the doc tool is now trying to create doc pages for scala.deriving.Mirror, which then forces the fromTuple method which has a refinement to MirroredElemTypes, causing the crash

sbt --client last projectC/Compile/doc
[info] entering *experimental* thin client - BEEP WHIRR
[info] terminate the server with `shutdown`
> last projectC/Compile/doc
[info] Main Scala API documentation to /Users/jamie/Workspace/scaladoc-tasty-err/project-C/target/scala-2.13/api...

[debug] Returning already retrieved and compiled bridge: /Users/jamie/.ivy2/local/org.scala-lang/scala2-sbt-bridge/2.13.14-bin-SNAPSHOT/jars/scala2-sbt-bridge.jar.
[debug] Calling Scaladoc with arguments:
[debug]     -d
[debug]     /Users/jamie/Workspace/scaladoc-tasty-err/project-C/target/scala-2.13/api
[debug]     -Ytasty-reader
[debug]     -bootclasspath
[debug]     /Users/jamie/.ivy2/local/org.scala-lang/scala-library/2.13.14-bin-SNAPSHOT/jars/scala-library.jar
[debug]     -classpath
[debug]     /Users/jamie/.ivy2/local/org.foo/base_3/0.1.0-SNAPSHOT/jars/base_3.jar:/Users/jamie/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.3.1/scala3-library_3-3.3.1.jar
[debug]     /Users/jamie/Workspace/scaladoc-tasty-err/project-C/src/main/scala/Main.scala
[info] Creating doc template for object Main
[info] Creating doc template for object Enum
[info] Creating doc template for trait Enum
[info] Creating doc template for object Mirror
[info] Creating doc template for class SingletonProxy
[info] Creating doc template for trait Singleton
[info] Creating doc template for trait Sum
[info] Creating doc template for trait Product
[info] Creating doc template for trait Mirror
[info] Creating doc template for object SourceFile
[info] Creating doc template for class SourceFile
[info] Creating doc template for object Child
[info] Creating doc template for class Child
[info] Creating doc template for class Tuple$package
[info] Creating doc template for class ADT
[error] stack trace is suppressed; run 'last projectC / Compile / doc' for the full output
[error] Total time: 0 s, completed Feb 23, 2024, 4:18:49 PM
[error] (projectC / Compile / doc) scala.reflect.internal.Types$TypeError: Unsupported Scala 3 generic tuple type scala.Tuple in bounds of type MirroredElemTypes; found in  scala.deriving.Mirror.<refinement>.

and on that old working commit https://github.com/scala/scala/commit/6b2c4aedc7d4d7d6a629b2ade6a6c04fe864ee14 we get instead

sbt --client last projectC/Compile/doc
[info] entering *experimental* thin client - BEEP WHIRR
[info] terminate the server with `shutdown`
> last projectC/Compile/doc
[info] Main Scala API documentation to /Users/jamie/Workspace/scaladoc-tasty-err/project-C/target/scala-2.13/api...

[debug] Calling Scaladoc with arguments:
[debug]     -d
[debug]     /Users/jamie/Workspace/scaladoc-tasty-err/project-C/target/scala-2.13/api
[debug]     -Ytasty-reader
[debug]     -bootclasspath
[debug]     /Users/jamie/.ivy2/local/org.scala-lang/scala-library/2.13.13-bin-SNAPSHOT/jars/scala-library.jar
[debug]     -classpath
[debug]     /Users/jamie/.ivy2/local/org.foo/base_3/0.1.0-SNAPSHOT/jars/base_3.jar:/Users/jamie/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.3.1/scala3-library_3-3.3.1.jar
[debug]     /Users/jamie/Workspace/scaladoc-tasty-err/project-C/src/main/scala/Main.scala
[info] Creating doc template for object Main
[info] Creating doc template for object SourceFile
[info] Creating doc template for class SourceFile
[info] Creating doc template for object Child
[info] Creating doc template for class Child
[info] Creating doc template for class ADT
[info] Main Scala API documentation successful.
[success] Total time: 0 s, completed Feb 23, 2024, 5:05:55 PM

Actually this looks wrong even in the "working" case. if I depend on the 2.13 compiled version of base then there should only be a doc page created for Main, there is no reason why ADT, Child etc are also getting made in https://github.com/scala/scala/commit/6b2c4aedc7d4d7d6a629b2ade6a6c04fe864ee14

bishabosha commented 3 months ago

Finally I have solved it.

Those were not being ignored because the sourceFile on the symbol was != null so was assumed to be a source file in the project. I had not added the case to ignore the associatedFile if it ends in ".tasty".