Cycle loading extends arguments from TASTy

BarkingBad commented 2 years ago

Compiler version

Minimized code

package bug

class RawXml(namespace2: String):
  class Value

object Body extends RawXml(Tei(null).namespace)

class Tei(text: Text):
  val namespace: String = ???

class Text(body: Body.Value):
  Text(null)

Then run:

bin/scalac -d here bug.scala

Output: compiles successfully

Then run:

bin/scalac -d here2 -from-tasty here/bug/Body.tasty here/bug/RawXml.tasty here/bug/Tei.tasty here/bug/Text.tasty

Output:

cannot take signature of MethodType(List(body), List(TypeRef(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object bug),Body),Value)), TypeRef(ThisType(TypeRef(NoPrefix,module class bug)),class Text))
cannot resolve reference to type bug.Body.type.Value
the classfile defining the type might be missing from the classpath while compiling here/bug/Body.tasty, here/bug/RawXml.tasty, here/bug/Tei.tasty, here/bug/Text.tasty
Exception in thread "main" dotty.tools.dotc.core.MissingType: 
        at dotty.tools.dotc.core.TypeErasure.dotty$tools$dotc$core$TypeErasure$$sigName(TypeErasure.scala:782)
        at dotty.tools.dotc.core.TypeErasure$.sigName(TypeErasure.scala:204)
        at dotty.tools.dotc.core.Signature.$anonfun$2(Signature.scala:113)
        at scala.collection.immutable.List.map(List.scala:246)
        at dotty.tools.dotc.core.Signature.prependTermParams(Signature.scala:113)
        at dotty.tools.dotc.core.Types$MethodOrPoly.computeSignature$2(Types.scala:3521)
        at dotty.tools.dotc.core.Types$MethodOrPoly.signature(Types.scala:3538)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.signature(Denotations.scala:612)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.signature(Denotations.scala:602)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.atSignature(Denotations.scala:644)
        at dotty.tools.dotc.core.Denotations$SingleDenotation.atSignature(Denotations.scala:642)
        at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readLengthTerm$1(TreeUnpickler.scala:1225)
        at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1299)
        at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readLengthTerm$1(TreeUnpickler.scala:1139)
        at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1299)
        at dotty.tools.dotc.core.tasty.TreeUnpickler.readRhs$1$$anonfun$1$$anonfun$1(TreeUnpickler.scala:810)
        at dotty.tools.dotc.core.tasty.TreeUnpickler$LazyReader.complete(TreeUnpickler.scala:1438)
        at dotty.tools.dotc.ast.Trees$WithLazyField.forceIfLazy(Trees.scala:1027)
        at dotty.tools.dotc.ast.Trees$WithLazyField.forceIfLazy$(Trees.scala:1022)
        at dotty.tools.dotc.ast.Trees$ValOrDefDef.forceIfLazy(Trees.scala:425)
        at dotty.tools.dotc.ast.Trees$ValOrDefDef.rhs(Trees.scala:430)
        at dotty.tools.dotc.ast.Trees$Instance$TreeAccumulator.foldOver(Trees.scala:1606)
        at dotty.tools.dotc.ast.Trees$Instance$TreeTraverser.traverseChildren(Trees.scala:1648)
        at dotty.tools.dotc.CompilationUnit$Force.traverse(CompilationUnit.scala:141)
        at dotty.tools.dotc.ast.Trees$Instance$TreeTraverser.apply(Trees.scala:1647)
        at dotty.tools.dotc.ast.Trees$Instance$TreeTraverser.apply(Trees.scala:1647)
        at dotty.tools.dotc.ast.Trees$Instance$TreeAccumulator.fold$1(Trees.scala:1522)
        at dotty.tools.dotc.ast.Trees$Instance$TreeAccumulator.apply(Trees.scala:1524)
        at dotty.tools.dotc.ast.Trees$Instance$TreeAccumulator.foldOver(Trees.scala:1617)
        at dotty.tools.dotc.ast.Trees$Instance$TreeTraverser.traverseChildren(Trees.scala:1648)
        at dotty.tools.dotc.CompilationUnit$Force.traverse(CompilationUnit.scala:141)
        at dotty.tools.dotc.ast.Trees$Instance$TreeTraverser.apply(Trees.scala:1647)
        at dotty.tools.dotc.ast.Trees$Instance$TreeTraverser.apply(Trees.scala:1647)
        at dotty.tools.dotc.ast.Trees$Instance$TreeAccumulator.foldOver(Trees.scala:1614)
        at dotty.tools.dotc.ast.Trees$Instance$TreeTraverser.traverseChildren(Trees.scala:1648)
        at dotty.tools.dotc.CompilationUnit$Force.traverse(CompilationUnit.scala:141)
        at dotty.tools.dotc.ast.Trees$Instance$TreeTraverser.apply(Trees.scala:1647)
        at dotty.tools.dotc.ast.Trees$Instance$TreeTraverser.apply(Trees.scala:1647)
        at dotty.tools.dotc.ast.Trees$Instance$TreeAccumulator.fold$1(Trees.scala:1522)
        at dotty.tools.dotc.ast.Trees$Instance$TreeAccumulator.apply(Trees.scala:1524)
        at dotty.tools.dotc.ast.Trees$Instance$TreeAccumulator.foldOver(Trees.scala:1623)
        at dotty.tools.dotc.ast.Trees$Instance$TreeAccumulator.foldOver(Trees.scala:1528)
        at dotty.tools.dotc.ast.Trees$Instance$TreeTraverser.traverseChildren(Trees.scala:1648)
        at dotty.tools.dotc.CompilationUnit$Force.traverse(CompilationUnit.scala:141)
        at dotty.tools.dotc.CompilationUnit$.apply(CompilationUnit.scala:105)
        at dotty.tools.dotc.CompilationUnit$.apply(CompilationUnit.scala:96)
        at dotty.tools.dotc.fromtasty.ReadTasty.compilationUnit$1(ReadTasty.scala:42)
        at dotty.tools.dotc.fromtasty.ReadTasty.readTASTY(ReadTasty.scala:70)
        at dotty.tools.dotc.fromtasty.ReadTasty.runOn$$anonfun$1(ReadTasty.scala:25)
        at scala.collection.immutable.List.flatMap(List.scala:293)
        at dotty.tools.dotc.fromtasty.ReadTasty.runOn(ReadTasty.scala:25)
        at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:261)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
        at dotty.tools.dotc.Run.runPhases$1(Run.scala:272)
        at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:280)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:68)
        at dotty.tools.dotc.Run.compileUnits(Run.scala:289)
        at dotty.tools.dotc.Run.compileUnits(Run.scala:228)
        at dotty.tools.dotc.fromtasty.TASTYRun.compile(TASTYRun.scala:12)
        at dotty.tools.dotc.Driver.doCompile(Driver.scala:39)
        at dotty.tools.dotc.Driver.process(Driver.scala:199)
        at dotty.tools.dotc.Driver.process(Driver.scala:167)
        at dotty.tools.dotc.Driver.process(Driver.scala:179)
        at dotty.tools.dotc.Driver.main(Driver.scala:209)
        at dotty.tools.dotc.Main.main(Main.scala)

Expectation

Compiles successfully

Disclaimer

I know this minimization is not very obvious but it is as most self-contained as I could achieve. The error comes from opentorah project and has been reported at Gradle. The sbt reproduction can be found at this branch https://github.com/BarkingBad/opentorah/blob/repro/base/src/main/scala/org/opentorah/tei/Body.scala though as you can see you can reproduce it using the compiler from dotty directly.

nicolasstucki commented 2 years ago

Nice minimization

nicolasstucki commented 2 years ago

The order in which they are loaded seems to be important. If I reverse the file list it works.

BarkingBad commented 2 years ago

Do you mean a list of tasty files?

EDIT: Ah, yes, indeed. I didn't see that

nicolasstucki commented 2 years ago

Slightly disentangled example

package bug

class Foo(x: Any)
trait RawXml:
  class Value

object Body extends Foo(Text.apply(null)), RawXml

object Text:
  def apply(body: Body.Value) = ???

This one fails when compiling Body alone from TASTy.

nicolasstucki commented 2 years ago

This test can be added to tests/pos/i13814.scala and run with testCompilation

odersky commented 2 years ago

I am not really sure how to fix this, since I have no experience with -from-tasty. Was there a specific reason to assign this to me?

Also: why does the title say "Cycle loading extends arguments from TASTy"? I don't see a cycle?

bishabosha commented 2 years ago

Also: why does the title say "Cycle loading extends arguments from TASTy"? I don't see a cycle?

object Body extends RawXml(Tei(null).namespace)

Tei takes a Text field which has a parameter of Body.Value (where Body.Value is inherited from RawXml)

nicolasstucki commented 2 years ago

@odersky, this is not a -from-tasty only issue anymore. It also happens in plain testCompilation during the pickling tests. scalac tests/pos/i13814.scala -Ytest-pickler with the example code is enough to trigger the issue.

As I understand it, when we unpickle Body we need to unpickle the extends clause to be able to know the members of Body. But to unpickle this extends we need to resolve Text.apply which requires Body.Value. But this member has not yet been added to Body as we haven't finished unpickling the extends clasue of Body and we haven't added RawXML as one of the parents.

odersky commented 2 years ago

Looks like a duplicate of #12872.

nicolasstucki commented 2 years ago

Indeed, looks like the cause is similar to #12872. But it fails in a different way, in #12872 we detect the cycle while here we fail to see a symbol that should be there while unpickling.

odersky commented 2 years ago

Ok, reassigning to @bishabosha together with #12872, as I won't have time to work on this for the next month or so.

scala / scala3