search

scala / scala3

The Scala 3 compiler, also known as Dotty.
https://dotty.epfl.ch
Apache License 2.0
5.79k stars 1.04k forks source link

Compiler crashes when combining lazy val and macros #21271

Open WojciechMazur opened 1 month ago

WojciechMazur commented 1 month ago

Similar to #20323 but this one is a regression introduced in 3.4.2 Based on OpenCB failure in shopstic/caliban - logs

Compiler version

Last good release: 3.4.2-RC1-bin-20240222-98efdab-NIGHTLY First bad release: 3.4.2-RC1-bin-20240226-e0cb1e7-NIGHTLY

Bisect points to 5a884bc3f8501e0bdf9ce35eb633147105d5a4f6 but compiler build deadlocked in a few revisions that were skipped, though it might not be accurate Can be any of changes between 98efdab..e0cb1e7

Minimized code

// schema.scala
trait Schema[-R, T]
object Schema extends GenericSchema[Any]

trait GenericSchema[R] extends SchemaDerivation[R]
trait SchemaDerivation[A]:
  lazy val sampleDate: String = ??? // lazy val requried to reproduce

  inline def derived[R, A]: Schema[R, A] =
    lazy val annotations = Macros.annotations[A]
    new Schema[R, A] {}

  inline given gen[R, A]: Schema[R, A] = derived[R, A]
// macros.scala
import scala.quoted.*

object Macros:
  inline def annotations[T]: List[Any] = ${ annotationsImpl[T] }
  def annotationsImpl[T: Type](using qctx: Quotes): Expr[List[Any]] = Expr.ofList(Nil)
// usage.scala
sealed trait InputValue

class Usage:
  val inputValueSchema: Schema[Any, InputValue] = Schema.gen

Output (click arrow to expand)

```scala unhandled exception while running MegaPhase{elimErasedValueType, pureStats, vcElideAllocations, etaReduce, arrayApply, elimPolyFunction, tailrec, completeJavaEnums, mixin, lazyVals, memoize, nonLocalReturns, capturedVars} on /Users/wmazur/projects/sandbox/src/main/scala/usage.scala An unhandled exception was thrown in the compiler. Please file a crash report here: https://github.com/lampepfl/dotty/issues/new/choose For non-enriched exceptions, compile with -Yno-enrich-error-messages. while compiling: /Users/wmazur/projects/sandbox/src/main/scala/usage.scala during phase: MegaPhase{elimErasedValueType, pureStats, vcElideAllocations, etaReduce, arrayApply, elimPolyFunction, tailrec, completeJavaEnums, mixin, lazyVals, memoize, nonLocalReturns, capturedVars} mode: Mode(ImplicitsEnabled) library version: version 2.13.12 compiler version: version 3.4.2-RC1-bin-SNAPSHOT-git-5a884bc settings: -classpath /Users/wmazur/.ivy2/local/org.scala-lang/scala3-library_3/3.4.2-RC1-bin-SNAPSHOT/jars/scala3-library_3.jar:/Users/wmazur/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.12/scala-library-2.13.12.jar -d /Users/wmazur/projects/sandbox/src/main/scala/.scala-build/scala_0de691b8c1/classes/main -java-output-version 17 -sourceroot /Users/wmazur/projects/sandbox/src/main/scala Exception while compiling /Users/wmazur/projects/sandbox/src/main/scala/macros.scala, /Users/wmazur/projects/sandbox/src/main/scala/test.scala, /Users/wmazur/projects/sandbox/src/main/scala/usage.scala An unhandled exception was thrown in the compiler. Please file a crash report here: https://github.com/lampepfl/dotty/issues/new/choose For non-enriched exceptions, compile with -Yno-enrich-error-messages. while compiling: during phase: parser mode: Mode() library version: version 2.13.12 compiler version: version 3.4.2-RC1-bin-SNAPSHOT-git-5a884bc settings: -classpath /Users/wmazur/.ivy2/local/org.scala-lang/scala3-library_3/3.4.2-RC1-bin-SNAPSHOT/jars/scala3-library_3.jar:/Users/wmazur/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.12/scala-library-2.13.12.jar -d /Users/wmazur/projects/sandbox/src/main/scala/.scala-build/scala_0de691b8c1/classes/main -java-output-version 17 -sourceroot /Users/wmazur/projects/sandbox/src/main/scala Exception in thread "main" dotty.tools.dotc.core.Denotations$StaleSymbolException: stale symbol; val OFFSET$_m_0#27281 in module class Schema$, defined in Period(2.78-86), is referred to in run Period(3.78) at dotty.tools.dotc.core.Denotations$SingleDenotation.staleSymbolError(Denotations.scala:960) at dotty.tools.dotc.core.Denotations$SingleDenotation.bringForward(Denotations.scala:758) at dotty.tools.dotc.core.Denotations$SingleDenotation.toNewRun$1(Denotations.scala:805) at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:876) at dotty.tools.dotc.core.Symbols$Symbol.recomputeDenot(Symbols.scala:124) at dotty.tools.dotc.core.Symbols$Symbol.computeDenot(Symbols.scala:118) at dotty.tools.dotc.core.Symbols$Symbol.denot(Symbols.scala:109) at dotty.tools.dotc.core.Symbols$.toDenot(Symbols.scala:542) at dotty.tools.dotc.transform.Mixin.traitInits$1$$anonfun$1(Mixin.scala:264) at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:479) at scala.collection.Iterator$$anon$9.hasNext(Iterator.scala:583) at scala.collection.immutable.List.prependedAll(List.scala:152) at scala.collection.immutable.List$.from(List.scala:684) at scala.collection.immutable.List$.from(List.scala:681) at scala.collection.IterableOps$WithFilter.map(Iterable.scala:898) at dotty.tools.dotc.transform.Mixin.traitInits$1(Mixin.scala:285) at dotty.tools.dotc.transform.Mixin.$anonfun$6(Mixin.scala:315) at scala.collection.immutable.List.flatMap(List.scala:293) at dotty.tools.dotc.transform.Mixin.transformTemplate(Mixin.scala:318) at dotty.tools.dotc.transform.Mixin.transformTemplate(Mixin.scala:184) at dotty.tools.dotc.transform.MegaPhase.goTemplate(MegaPhase.scala:1067) at dotty.tools.dotc.transform.MegaPhase.goTemplate(MegaPhase.scala:1068) at dotty.tools.dotc.transform.MegaPhase.goTemplate(MegaPhase.scala:1068) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:373) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:450) at dotty.tools.dotc.transform.MegaPhase.transformNamed$1(MegaPhase.scala:268) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:448) at dotty.tools.dotc.transform.MegaPhase.loop$2(MegaPhase.scala:467) at dotty.tools.dotc.transform.MegaPhase.transformBlock(MegaPhase.scala:472) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:311) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:450) at dotty.tools.dotc.transform.MegaPhase.loop$2$$anonfun$1(MegaPhase.scala:470) at dotty.tools.dotc.transform.MegaPhase.loop$2(MegaPhase.scala:472) at dotty.tools.dotc.transform.MegaPhase.transformBlock(MegaPhase.scala:472) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:311) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:450) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:333) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:450) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:333) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:450) at dotty.tools.dotc.transform.MegaPhase.loop$2$$anonfun$1(MegaPhase.scala:470) at dotty.tools.dotc.transform.MegaPhase.loop$2(MegaPhase.scala:472) at dotty.tools.dotc.transform.MegaPhase.transformBlock(MegaPhase.scala:472) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:311) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:450) at dotty.tools.dotc.transform.MegaPhase.mapDefDef$1(MegaPhase.scala:261) at dotty.tools.dotc.transform.MegaPhase.transformNamed$1(MegaPhase.scala:264) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:448) at dotty.tools.dotc.transform.MegaPhase.loop$1(MegaPhase.scala:461) at dotty.tools.dotc.transform.MegaPhase.transformStats(MegaPhase.scala:461) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:372) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:450) at dotty.tools.dotc.transform.MegaPhase.transformNamed$1(MegaPhase.scala:268) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:448) at dotty.tools.dotc.transform.MegaPhase.loop$1(MegaPhase.scala:461) at dotty.tools.dotc.transform.MegaPhase.transformStats(MegaPhase.scala:461) at dotty.tools.dotc.transform.MegaPhase.mapPackage$1(MegaPhase.scala:392) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:395) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:450) at dotty.tools.dotc.transform.MegaPhase.transformUnit(MegaPhase.scala:477) at dotty.tools.dotc.transform.MegaPhase.run(MegaPhase.scala:489) at dotty.tools.dotc.core.Phases$Phase.runOn$$anonfun$1(Phases.scala:354) at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15) at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10) at scala.collection.immutable.List.foreach(List.scala:333) at dotty.tools.dotc.core.Phases$Phase.runOn(Phases.scala:360) at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:315) at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15) at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10) at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323) at dotty.tools.dotc.Run.runPhases$1(Run.scala:337) at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:350) at dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:360) at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:69) at dotty.tools.dotc.Run.compileUnits(Run.scala:360) at dotty.tools.dotc.Run.compileUnits(Run.scala:267) at dotty.tools.dotc.Run.compileSuspendedUnits(Run.scala:371) at dotty.tools.dotc.Driver.finish(Driver.scala:57) at dotty.tools.dotc.Driver.doCompile(Driver.scala:38) at dotty.tools.dotc.Driver.process(Driver.scala:196) at dotty.tools.dotc.Driver.process(Driver.scala:164) at dotty.tools.dotc.Driver.process(Driver.scala:176) at dotty.tools.dotc.Driver.main(Driver.scala:206) at dotty.tools.dotc.Main.main(Main.scala) ```
KacperFKorban commented 1 month ago

@Gedochao Do you mind if I reassign this to myself? (@jchyb is away right now anyway) Plus this seems unrelated to metaprogramming to me. It seems to me that the only thing the macro expansion changes is incrementing the runId for the usage file. Also, the affecting PR is this one: https://github.com/scala/scala3/pull/19786, specifically the recomputeDenot change

Also', here is a smaller minimization:

// Macro.scala
import scala.quoted.*

trait Schema
object Schema:
  lazy val sampleDate: String = "" // lazy val requried to reproduce

  inline def derived: Schema =
    annotations
    new Schema {}

inline def annotations: Int = ${ annotationsImpl }
def annotationsImpl(using Quotes): Expr[Int] = Expr(1)
// Test.scala
//> using scala 3.nightly

val inputValueSchema = Schema.derived

Output:

```scala -- [E197] Potential Issue Warning: ~/bugs/i21271/Macro.scala:10:4 ------ 10 | new Schema {} | ^ | New anonymous class definition will be duplicated at each inline site | | longer explanation available when compiling with `-explain` 1 warning found unhandled exception while running MegaPhase{elimErasedValueType, pureStats, vcElideAllocations, etaReduce, arrayApply, elimPolyFunction, tailrec, completeJavaEnums, mixin, lazyVals, memoize, nonLocalReturns, capturedVars} on ~/bugs/i21271/Test.scala An unhandled exception was thrown in the compiler. Please file a crash report here: https://github.com/scala/scala3/issues/new/choose For non-enriched exceptions, compile with -Xno-enrich-error-messages. while compiling: ~/bugs/i21271/Test.scala during phase: MegaPhase{elimErasedValueType, pureStats, vcElideAllocations, etaReduce, arrayApply, elimPolyFunction, tailrec, completeJavaEnums, mixin, lazyVals, memoize, nonLocalReturns, capturedVars} mode: Mode(ImplicitsEnabled) library version: version 2.13.14 compiler version: version 3.6.0-RC1-bin-20240724-f0b3a2b-NIGHTLY-git-f0b3a2b settings: -classpath ~/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.6.0-RC1-bin-20240724-f0b3a2b-NIGHTLY/scala3-library_3-3.6.0-RC1-bin-20240724-f0b3a2b-NIGHTLY.jar:~/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.14/scala-library-2.13.14.jar -d ~/bugs/i21271/.scala-build/i21271_73afa7fde9/classes/main -java-output-version 17 -sourceroot ~/bugs/i21271 Exception while compiling ~/bugs/i21271/Macro.scala, ~/bugs/i21271/Test.scala An unhandled exception was thrown in the compiler. Please file a crash report here: https://github.com/scala/scala3/issues/new/choose For non-enriched exceptions, compile with -Xno-enrich-error-messages. while compiling: during phase: parser mode: Mode() library version: version 2.13.14 compiler version: version 3.6.0-RC1-bin-20240724-f0b3a2b-NIGHTLY-git-f0b3a2b settings: -classpath ~/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.6.0-RC1-bin-20240724-f0b3a2b-NIGHTLY/scala3-library_3-3.6.0-RC1-bin-20240724-f0b3a2b-NIGHTLY.jar:~/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.14/scala-library-2.13.14.jar -d ~/bugs/i21271/.scala-build/i21271_73afa7fde9/classes/main -java-output-version 17 -sourceroot ~/bugs/i21271 Exception in thread "main" dotty.tools.dotc.core.Denotations$StaleSymbolException: stale symbol; val OFFSET$_m_0#36284 in module class Schema$, defined in Period(2.78-86), is referred to in run Period(3.78) at dotty.tools.dotc.core.Denotations$SingleDenotation.staleSymbolError(Denotations.scala:961) at dotty.tools.dotc.core.Denotations$SingleDenotation.bringForward(Denotations.scala:759) at dotty.tools.dotc.core.Denotations$SingleDenotation.toNewRun$1(Denotations.scala:806) at dotty.tools.dotc.core.Denotations$SingleDenotation.current(Denotations.scala:877) at dotty.tools.dotc.core.Symbols$Symbol.recomputeDenot(Symbols.scala:124) at dotty.tools.dotc.core.Symbols$Symbol.computeDenot(Symbols.scala:118) at dotty.tools.dotc.core.Symbols$Symbol.denot(Symbols.scala:109) at dotty.tools.dotc.core.Symbols$.toDenot(Symbols.scala:544) at dotty.tools.dotc.transform.Mixin.traitInits$1$$anonfun$1(Mixin.scala:266) at scala.collection.Iterator$$anon$6.hasNext(Iterator.scala:479) at scala.collection.Iterator$$anon$9.hasNext(Iterator.scala:583) at scala.collection.immutable.List.prependedAll(List.scala:152) at scala.collection.immutable.List$.from(List.scala:685) at scala.collection.immutable.List$.from(List.scala:682) at scala.collection.IterableOps$WithFilter.map(Iterable.scala:900) at dotty.tools.dotc.transform.Mixin.traitInits$1(Mixin.scala:265) at dotty.tools.dotc.transform.Mixin.$anonfun$6(Mixin.scala:325) at scala.collection.immutable.List.flatMap(List.scala:294) at dotty.tools.dotc.transform.Mixin.transformTemplate(Mixin.scala:320) at dotty.tools.dotc.transform.Mixin.transformTemplate(Mixin.scala:186) at dotty.tools.dotc.transform.MegaPhase.goTemplate(MegaPhase.scala:1071) at dotty.tools.dotc.transform.MegaPhase.goTemplate(MegaPhase.scala:1072) at dotty.tools.dotc.transform.MegaPhase.goTemplate(MegaPhase.scala:1072) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:377) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:454) at dotty.tools.dotc.transform.MegaPhase.transformNamed$1(MegaPhase.scala:272) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:452) at dotty.tools.dotc.transform.MegaPhase.loop$2(MegaPhase.scala:471) at dotty.tools.dotc.transform.MegaPhase.transformBlock(MegaPhase.scala:476) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:315) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:454) at dotty.tools.dotc.transform.MegaPhase.loop$2$$anonfun$1(MegaPhase.scala:474) at dotty.tools.dotc.transform.MegaPhase.loop$2(MegaPhase.scala:476) at dotty.tools.dotc.transform.MegaPhase.transformBlock(MegaPhase.scala:476) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:315) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:454) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:337) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:454) at dotty.tools.dotc.transform.MegaPhase.mapDefDef$1(MegaPhase.scala:265) at dotty.tools.dotc.transform.MegaPhase.transformNamed$1(MegaPhase.scala:268) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:452) at dotty.tools.dotc.transform.MegaPhase.loop$1(MegaPhase.scala:465) at dotty.tools.dotc.transform.MegaPhase.transformStats(MegaPhase.scala:465) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:376) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:454) at dotty.tools.dotc.transform.MegaPhase.transformNamed$1(MegaPhase.scala:272) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:452) at dotty.tools.dotc.transform.MegaPhase.loop$1(MegaPhase.scala:465) at dotty.tools.dotc.transform.MegaPhase.transformStats(MegaPhase.scala:465) at dotty.tools.dotc.transform.MegaPhase.mapPackage$1(MegaPhase.scala:396) at dotty.tools.dotc.transform.MegaPhase.transformUnnamed$1(MegaPhase.scala:399) at dotty.tools.dotc.transform.MegaPhase.transformTree(MegaPhase.scala:454) at dotty.tools.dotc.transform.MegaPhase.transformUnit(MegaPhase.scala:481) at dotty.tools.dotc.transform.MegaPhase.run(MegaPhase.scala:493) at dotty.tools.dotc.core.Phases$Phase.runOn$$anonfun$1(Phases.scala:380) at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15) at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10) at scala.collection.immutable.List.foreach(List.scala:334) at dotty.tools.dotc.core.Phases$Phase.runOn(Phases.scala:373) at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:343) at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15) at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10) at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323) at dotty.tools.dotc.Run.runPhases$1(Run.scala:336) at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:384) at dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:396) at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:69) at dotty.tools.dotc.Run.compileUnits(Run.scala:396) at dotty.tools.dotc.Run.compileUnits(Run.scala:288) at dotty.tools.dotc.Run.compileSuspendedUnits(Run.scala:410) at dotty.tools.dotc.Driver.finish(Driver.scala:63) at dotty.tools.dotc.Driver.doCompile(Driver.scala:38) at dotty.tools.dotc.Driver.process(Driver.scala:201) at dotty.tools.dotc.Driver.process(Driver.scala:169) at dotty.tools.dotc.Driver.process(Driver.scala:181) at dotty.tools.dotc.Driver.main(Driver.scala:211) at dotty.tools.dotc.Main.main(Main.scala) Compilation failed ```
jchyb commented 1 month ago

When a macro definition and call are located in the same compilation run, the compiler suspends the macro call compilation unit, finishes compiling other stuff and comes back to the macro later. Some parts of the context and symbol denotations can be shared between those runs which can cause those stale symbol issues. I believe there is might be a pass refreshing denotations somewhere in typer for exactly that occasion, but I still don't quite understand everything there. With that said please feel free to try taking this on if you want @KacperFKorban, especially since you have a lead. I find those stale symbol errors very difficult personally, so any help is appreciated