Closed noresttherein closed 1 year ago
Probably it's the post-typer checks, unit.toCheck
, that is getting updated during iteration.
There was just an unrelated improvement in that mechanism.
ha, I haven't seen this one before either.
the code in question (in Analyzer.scala
), for (workItem <- unit.toCheck) workItem()
, where toCheck
is a ListBuffer
of thunks and there are various sites elsewhere where we do unit.toCheck += ...
, fails thanks to https://github.com/scala/scala/pull/9174 (merged in Scala 2.13.4)
I get to say jinx before Seth does?
I owe @som-snytt a Coke
it looks like we could make the code in question safe pretty easily, and without noticeably impacting performance (it's very much an outer loop), by replacing
for (workItem <- unit.toCheck) workItem()
with something like
while (unit.toCheck.nonEmpty) {
val items = unit.toCheck.toList
unit.toCheck.clear()
for (workItem <- items) workItem()
}
that kind of thing
I am swamped this week, perhaps someone else would like to jump on it first?
These are checks run (currently) after a file is typechecked (and just before unused warnings), so it should (perhaps) be obvious from debug output which file was just typechecked. A reproduction would be nice. I'll volunteer the easy fix as proposed.
Yes, I do have output giving a strong indication of what has been just typechecked. But I have no idea what can I do to improve this issue on my side? Of course, as far as reproduction goes, I can link a commit in my project which fails, but
Can I somehow compile just those two files against current 2.13.10 and put them in bootclasspath?
I don't know the simplest way to put a couple of class files on sbt's class path, if that is the question.
Usually I wind up publishing a local jar to include in the build.
I will try to enumerate when it adds "post-typer" work items and which might incur such tasks recursively. I think if it tries to add a work item after typer has run, it should just execute it immediately. (I tried the enumeration yesterday but fell asleep.)
I was asking if the two files will pose any conflicts with the existing code base, or can I just replace those two without making my own version of the compiler, but I see I can't, there is a ClassCastException
.
Once there's an open PR, you'll be able to test the PR validation snapshot; and once the PR is merged, you'll be able to use the next nightly.
Out of the fire and into the frying pan... I cloned @som-snytt's repo and built from the branch with the fix. Now the compiler just hangs indefinitely on the same file (uses a lot of CPU, so in an infinite loop, looks like it). More precisely, it does not print anything for the file the processing of which was throwing the exception, just the one before it.
A JVM thread dump would show where in the compiler code the cycle is.
Perhaps the same file is getting adding to toCheck
over and over again forever; a println
would diagnose that.
It doesn't try to run "extra" checks but errors. (I haven't concocted a test yet.)
If @noresttherein points me to a repo, I'd be happy to give it a shot. The PR includes extra enhancements, any breakage is possible.
I see activity on sugar
, is the problematic state pushed to the repo?
Also, are you quite sure you don't want to call it sugah
?
Sorry, it was getting late yesterday and I just didn't have the strength to tackle it.
I attached a dubugger to @som-snytt's commit branch for this issue, and the recursion in isSubtype
goes as follows:
tp1=_
, tp2=_
tp1=_
, tp2=IndexedMapping[_, Unit]#Subject
tp1=IndexedMapping[_, Unit]#Subject
, tp2=_
tp1=IndexedMapping[_, Unit]#Subject
, tp2=IndexedMapping[_, Unit]#Subject
tp1=_
, tp2=_
It is important to know that:
trait IndexedMapping[S, O] extends ... {
type Subject = S
...
}
I suspect there may be an abstract type projection going on here.
However, it is not a simple 4-step recursion: these four parameter combinations, and no other, repeat themselves, but in varying order, and I have troubles noticing a pattern. If anyone has an idea how I can do a stack dump together with values of all parameters (including AnyRef
types on the heap), in a readable format, I'd be happy to do it.
Here is the stack trace of the compiler thread:
"NGSession 1: 127.0.0.1: compile@6954" tid=0x17 nid=NA runnable
java.lang.Thread.State: RUNNABLE
at scala.reflect.internal.tpe.TypeComparers.isSubType(TypeComparers.scala:268)
at scala.reflect.internal.tpe.TypeComparers.isSubType$(TypeComparers.scala:267)
at scala.reflect.internal.SymbolTable.isSubType(SymbolTable.scala:28)
at scala.reflect.internal.Types$Type.$less$colon$less(Types.scala:809)
at scala.reflect.internal.Types$TypeBounds.containsType(Types.scala:1551)
at scala.reflect.internal.Types.$anonfun$isWithinBounds$4(Types.scala:4978)
at scala.reflect.internal.Types.$anonfun$isWithinBounds$4$adapted(Types.scala:4977)
at scala.reflect.internal.Types$$Lambda$1127/0x00000008014f6dc0.apply(Unknown Source:-1)
at scala.collection.immutable.List.corresponds(List.scala:437)
at scala.reflect.internal.Types.isWithinBounds(Types.scala:4977)
at scala.reflect.internal.Types.isWithinBounds$(Types.scala:4968)
at scala.reflect.internal.SymbolTable.isWithinBounds(SymbolTable.scala:28)
at scala.reflect.internal.Types$ExistentialType.withTypeVars(Types.scala:3207)
at scala.reflect.internal.tpe.TypeComparers.thirdTry$1(TypeComparers.scala:552)
at scala.reflect.internal.tpe.TypeComparers.secondTry$1(TypeComparers.scala:519)
at scala.reflect.internal.tpe.TypeComparers.firstTry$1(TypeComparers.scala:495)
at scala.reflect.internal.tpe.TypeComparers.isSubType2(TypeComparers.scala:615)
at scala.reflect.internal.tpe.TypeComparers.isSubType1(TypeComparers.scala:347)
at scala.reflect.internal.tpe.TypeComparers.isSubType(TypeComparers.scala:305)
at scala.reflect.internal.tpe.TypeComparers.isSubType$(TypeComparers.scala:267)
at scala.reflect.internal.SymbolTable.isSubType(SymbolTable.scala:28)
at scala.reflect.internal.Types.isSubArg$1(Types.scala:4765)
at scala.reflect.internal.Types.$anonfun$isSubArgs$1(Types.scala:4768)
at scala.reflect.internal.Types.$anonfun$isSubArgs$1$adapted(Types.scala:4768)
at scala.reflect.internal.Types$$Lambda$1316/0x000000080153fda0.apply(Unknown Source:-1)
at scala.reflect.internal.util.Collections.corresponds3(Collections.scala:33)
at scala.reflect.internal.util.Collections.corresponds3$(Collections.scala:30)
at scala.reflect.internal.SymbolTable.corresponds3(SymbolTable.scala:28)
at scala.reflect.internal.Types.isSubArgs(Types.scala:4768)
at scala.reflect.internal.Types.isSubArgs$(Types.scala:4762)
at scala.reflect.internal.SymbolTable.isSubArgs(SymbolTable.scala:28)
at scala.reflect.internal.tpe.TypeComparers.firstTry$1(TypeComparers.scala:470)
at scala.reflect.internal.tpe.TypeComparers.isSubType2(TypeComparers.scala:615)
at scala.reflect.internal.tpe.TypeComparers.isSubType1(TypeComparers.scala:347)
at scala.reflect.internal.tpe.TypeComparers.isSubType(TypeComparers.scala:305)
at scala.reflect.internal.tpe.TypeComparers.isSubType$(TypeComparers.scala:267)
at scala.reflect.internal.SymbolTable.isSubType(SymbolTable.scala:28)
at scala.reflect.internal.Types$Type.$less$colon$less(Types.scala:809)
at scala.tools.nsc.typechecker.Infer$Inferencer.inferTypedPattern(Infer.scala:1281)
at scala.tools.nsc.typechecker.PatternTypers$PatternTyper.typedInPattern(PatternTypers.scala:194)
at scala.tools.nsc.typechecker.PatternTypers$PatternTyper.typedInPattern$(PatternTypers.scala:181)
at scala.tools.nsc.typechecker.Typers$Typer.typedInPattern(Typers.scala:203)
at scala.tools.nsc.typechecker.Typers$Typer.typedTyped$1(Typers.scala:5763)
at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:6027)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:6041)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6080)
at scala.tools.nsc.typechecker.Typers$Typer.typedBind$1(Typers.scala:4663)
at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:6023)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:6041)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6080)
at scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedPattern$2(Typers.scala:6259)
at scala.tools.nsc.typechecker.Typers$Typer$$Lambda$1145/0x00000008014fa930.apply(Unknown Source:-1)
at scala.tools.nsc.typechecker.Contexts$Context.withMode(Contexts.scala:609)
at scala.tools.nsc.typechecker.Contexts$Context.withImplicitsDisabledAllowEnrichment(Contexts.scala:615)
at scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedPattern$1(Typers.scala:6259)
at scala.tools.nsc.typechecker.Typers$Typer$$Lambda$1144/0x00000008014fa688.apply(Unknown Source:-1)
at scala.tools.nsc.typechecker.TypeDiagnostics.typingInPattern(TypeDiagnostics.scala:71)
at scala.tools.nsc.typechecker.TypeDiagnostics.typingInPattern$(TypeDiagnostics.scala:68)
at scala.tools.nsc.Global$$anon$6.typingInPattern(Global.scala:511)
at scala.tools.nsc.typechecker.Typers$Typer.typedPattern(Typers.scala:6259)
at scala.tools.nsc.typechecker.Typers$Typer.typedCase(Typers.scala:2604)
at scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedCases$1(Typers.scala:2638)
at scala.tools.nsc.typechecker.Typers$Typer$$Lambda$1141/0x00000008014ff680.apply(Unknown Source:-1)
at scala.collection.immutable.List.loop$3(List.scala:472)
at scala.collection.immutable.List.mapConserve(List.scala:497)
at scala.tools.nsc.typechecker.Typers$Typer.typedCases(Typers.scala:2637)
at scala.tools.nsc.typechecker.Typers$Typer.typedMatch(Typers.scala:2649)
at scala.tools.nsc.typechecker.Typers$Typer.typedVirtualizedMatch$1(Typers.scala:4805)
at scala.tools.nsc.typechecker.Typers$Typer.typedOutsidePatternMode$1(Typers.scala:6003)
at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:6034)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:6041)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6080)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6169)
at scala.tools.nsc.typechecker.Typers$Typer.typedCase(Typers.scala:2619)
at scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedCases$1(Typers.scala:2638)
at scala.tools.nsc.typechecker.Typers$Typer$$Lambda$1141/0x00000008014ff680.apply(Unknown Source:-1)
at scala.collection.immutable.List.loop$3(List.scala:472)
at scala.collection.immutable.List.mapConserve(List.scala:497)
at scala.tools.nsc.typechecker.Typers$Typer.typedCases(Typers.scala:2637)
at scala.tools.nsc.typechecker.Typers$Typer.typedMatch(Typers.scala:2649)
at scala.tools.nsc.typechecker.Typers$Typer.typedVirtualizedMatch$1(Typers.scala:4805)
at scala.tools.nsc.typechecker.Typers$Typer.typedOutsidePatternMode$1(Typers.scala:6003)
at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:6034)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:6041)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6080)
at scala.tools.nsc.typechecker.Typers$Typer.typedBlock(Typers.scala:2579)
at scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typed1$108(Typers.scala:5998)
at scala.tools.nsc.typechecker.Typers$Typer$$Lambda$955/0x00000008014b2660.apply(Unknown Source:-1)
at scala.tools.nsc.typechecker.Typers$Typer.typerWithLocalContext(Typers.scala:501)
at scala.tools.nsc.typechecker.Typers$Typer.typedOutsidePatternMode$1(Typers.scala:5998)
at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:6034)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:6041)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6080)
at scala.tools.nsc.typechecker.Typers$Typer.transformedOrTyped(Typers.scala:6344)
at scala.tools.nsc.typechecker.Typers$Typer.typedDefDef(Typers.scala:2400)
at scala.tools.nsc.typechecker.Typers$Typer.typedMemberDef$1(Typers.scala:5986)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:6040)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6080)
at scala.tools.nsc.typechecker.Typers$Typer.typedByValueExpr(Typers.scala:6158)
at scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:3279)
at scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedStats$8(Typers.scala:3426)
at scala.tools.nsc.typechecker.Typers$Typer$$Lambda$931/0x00000008014ac670.apply(Unknown Source:-1)
at scala.collection.immutable.List.loop$3(List.scala:472)
at scala.collection.immutable.List.mapConserve(List.scala:497)
at scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3426)
at scala.tools.nsc.typechecker.Typers$Typer.typedTemplate(Typers.scala:2073)
at scala.tools.nsc.typechecker.Typers$Typer.typedClassDef(Typers.scala:1911)
at scala.tools.nsc.typechecker.Typers$Typer.typedMemberDef$1(Typers.scala:5987)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:6040)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6080)
at scala.tools.nsc.typechecker.Typers$Typer.typedByValueExpr(Typers.scala:6158)
at scala.tools.nsc.typechecker.Typers$Typer.typedStat$1(Typers.scala:3279)
at scala.tools.nsc.typechecker.Typers$Typer.$anonfun$typedStats$8(Typers.scala:3426)
at scala.tools.nsc.typechecker.Typers$Typer$$Lambda$931/0x00000008014ac670.apply(Unknown Source:-1)
at scala.collection.immutable.List.loop$3(List.scala:472)
at scala.collection.immutable.List.mapConserve(List.scala:497)
at scala.tools.nsc.typechecker.Typers$Typer.typedStats(Typers.scala:3426)
at scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:5670)
at scala.tools.nsc.typechecker.Typers$Typer.typedMemberDef$1(Typers.scala:5990)
at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:6040)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6080)
at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:6156)
at scala.tools.nsc.typechecker.Analyzer$typerFactory$TyperPhase.apply(Analyzer.scala:127)
at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:480)
at scala.tools.nsc.typechecker.Analyzer$typerFactory$TyperPhase.run(Analyzer.scala:110)
at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1543)
at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1527)
at scala.tools.nsc.Global$Run.compileSources(Global.scala:1519)
at scala.tools.nsc.Global$Run.compileFiles(Global.scala:1632)
at xsbt.CachedCompiler0.run(CompilerBridge.scala:163)
at xsbt.CachedCompiler0.run(CompilerBridge.scala:134)
How to make a collapsible element here?
Sorry, concurrent posts. No, sugar
is very well-behaved.
The one written in my blood and tears is https://github.com/noresttherein/oldsql/. Branch scalac-crash-12726
.
As I mentioned early on, it is an end of a refactor and I am inching forward through countless errors, so sorry about that.
I named the version from your commit 2.13.10-patch
, so you'll have to put the artifacts in your repo.
Now that I know the file (SelectSQL
) and have a lead, I'll try to find the problematic spot. Hopefully it is an error on my side (although of course it would be nice if the compiler reacted differently).
I am not sure however that it is even related to this issue, although I find it suspect that neither of the lists of processed (i.e. listed warnings and errors) files in pure 2.13.10
and the patch are a prefix of the other - in official 2.13.10
the last output was for RelationSQL
, and, judging by typer logs, it was done with it.
Thanks, I'll take a casual look. I know lrytz likes to debugger the compiler, so maybe I will go down that path.
Your project link is lunked. (I just made up that word. I was redeploying lunk as in lunkhead, but there's also a word lunker for a big fish, TIL.)
Edit: by lunked, I mean the URL is munged in the link.
I can reproduce the CME against HEAD easily, now I'll try the supposed fix.
I also had to search for markdown collapse, after searching in vain for an old dotty ticket to demonstrate it. Use details tag with summary tag inside for description.
Also no idea what this shorter log message means.
Last few lines of logging
I pushed the other pathologies to the linked ticket, and I'll take another look at what causes the original CME here.
Of course, a post-typer task that typechecks can incur arbitrary work. In this case, typedRefinement
caused an additional feature check. In sum, not very mysterious.
UNIT WORK RelationSQL.scala has 105 ITEMS
java.lang.Throwable: ADD POST UNIT on Thread[#103,pool-6-thread-5,5,main]
at scala.tools.nsc.CompilationUnits$CompilationUnit.addPostUnitCheck(CompilationUnits.scala:132)
at scala.tools.nsc.typechecker.Typers$Typer.checkFeature(Typers.scala:753)
at scala.tools.nsc.typechecker.Typers$Typer.checkExistentialsFeature(Typers.scala:760)
at scala.tools.nsc.typechecker.Typers$Typer.typedExistentialTypeTree$1(Typers.scala:5926)
Scala 2.13.10
Now this is something I haven't seen.
StackOverflowError
sure, half a dozen a day, I can understand them. But this?I haven't a faintest idea in what class, file or even package this happens, so I have no way of extracting it. Maybe it will tell something to someone. Any ideas of how to look for clues? I have about every logging option enabled which doesn't throw an
AssertionError
, orNullPointerException
itself. Unfortunately, the last line from-Ylog:0-100
seems to be from a successfully processed file (I got a summary of warnings for the file), so I am at a loss.