Closed scabug closed 9 years ago
scabug
commented
9 years ago Imported From: https://issues.scala-lang.org/browse/SI-9032?orig=1 Reporter: Robby (robby) Affected Versions: 2.11.4 See #8999 Attachments:
scabug
commented
9 years ago @retronym said: Hopefully this turns out to be the same regression as #8999. https://github.com/scala/scala/pull/4193 proposes a fix for that bug.
I haven't had a change to test this theory out yet, but will mark this bug as a blocker for 2.11.5 to make sure we do.
scabug
commented
9 years ago @retronym said: @gbasler do you have time to verify your fix against this build?
scabug
commented
9 years ago @gbasler said (edited on Dec 10, 2014 10:38:15 AM UTC): Yes, the fix solves the problem (see below).
Scala 2.11.4 took over 11 GB to compile:
[success] Total time: 2383 s, completed 10.12.2014 10:28:25
D:\code\sireum-build\sireum-build>Scala 2.11.5 + fix used about 1.4 GB:
[success] Total time: 148 s, completed 10.12.2014 11:02:56
D:\code\sireum-build\sireum-build>2.11.4 (partial) profile shows that a lot of time is spent in the pattern macher:
CPU hot spots
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+
| Name | Time (ms) |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+
| +---scala.tools.nsc.transform.patmat.MatchAnalysis$MatchAnalyzer$class.modelToCounterExample(MatchAnalysis$MatchAnalyzer, ScalaLogic$TreesAndTypesDomain$Var, Map) | 17'113 100 % |
| | | |
| +---scala.collection.TraversableLike$$anonfun$filterImpl$1.apply(Object) | 15'272 89 % |
| | | |
| +---scala.tools.nsc.transform.patmat.MatchAnalysis$MatchAnalyzer$VariableAssignment$3$$anonfun$uniqueEqualTo$1.apply(ScalaLogic$TreesAndTypesDomain$Const) | 14'430 84 % |
| | | |
| +---scala.collection.LinearSeqOptimized$class.exists(LinearSeqOptimized, Function1) | 14'383 84 % |
| | | |
| +---scala.reflect.internal.tpe.TypeComparers$class.isSubType(SymbolTable, Types$Type, Types$Type, int) | 8'517 50 % |
| | | |
| +---scala.reflect.internal.tpe.TypeComparers$class.firstTry$1(SymbolTable, Types$Type, Types$Type, int) | 7'020 41 % |
| | | |
| +---scala.reflect.internal.tpe.TypeComparers$class.isSameType(SymbolTable, Types$Type, Types$Type) | 5'257 31 % |
| | | |
| +---scala.reflect.internal.tpe.TypeComparers$class.isSameType2(SymbolTable, Types$Type, Types$Type) | 4'726 28 % |
| | | |
| +---scala.reflect.internal.Types$CompoundType.baseType(Symbols$Symbol) | 4'024 24 % |
| | | |
| +---scala.reflect.internal.Types$Type.baseTypeIndex(Symbols$Symbol) | 3'993 23 % |
| | | |
| +---scala.reflect.internal.Types$class.normalizePlus(SymbolTable, Types$Type) | 3'400 20 % |
| | | |
| +---scala.reflect.internal.SymbolTable.isRawType(Types$Type) | 3'354 20 % |
| | | |
| +---scala.reflect.internal.Types$class.isRawType(SymbolTable, Types$Type) | 3'198 19 % |
| | | |
| +---scala.reflect.internal.Symbols$Symbol.baseTypeSeqLength$1(Symbols$Symbol) | 3'135 18 % |
| | | |
| +---scala.reflect.internal.Symbols$Symbol.info() | 2'293 13 % |
| | | |
| +---scala.reflect.internal.Types$class.isRawIfWithoutArgs(SymbolTable, Symbols$Symbol) | 1'638 10 % |
| | | |
| +---scala.reflect.internal.Symbols$Symbol.isMonomorphicType() | 1'591 9 % |
| | | |
| +---scala.reflect.internal.Symbols$Symbol.rawInfo() | 1'326 8 % |
| | | |
| +---scala.reflect.internal.tpe.TypeComparers$class.thirdTryRef$1(SymbolTable, Types$Type, Types$TypeRef, Types$Type, Types$Type, int) | 1'279 7 % |
| | | |
| +---scala.reflect.internal.tpe.TypeComparers$class.classOnRight$1(SymbolTable, Types$Type, Types$Type, int, Types$Type, Types$TypeRef, Symbols$Symbol) | 1'216 7 % |
| | | |
| +---scala.reflect.internal.Types$NoArgsTypeRef.typeParams() | 1'107 6 % |
| | | |
| +---scala.collection.LinearSeqOptimized$class.contains(LinearSeqOptimized, Object) | 920 5 % |
| | | |
| +---scala.reflect.internal.Types$TypeRef.normalize() | 920 5 % |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+
Generated by YourKit Java Profiler 2013 build 13088 10.12.2014 11:06:56@gbasler said: Attached SBT logs.
scabug
commented
9 years ago Robby (robby) said: Good news. Thanks for looking into this.
So, how do I access 2.11.5-fix-oom to try it out?
scabug
commented
9 years ago @gbasler said: You'd have to check out my branch (see above), then compile it:
ant publish-local-opt -Dmaven.version.suffix="-fix-oom"and modify your build:
final val scalaVer = "2.11.5-fix-oom"
...
resolvers += "Local Maven Repository" at "file:///"+Path.userHome+"/.m2/repository"
...
libraryDependencies += "org.scalatest" % "scalatest_2.11" % "2.2.1" % "test",Then you can compile it with SBT.
scabug
commented
9 years ago Robby (robby) said: Great! I can confirm that the proposed fix resolves the issue (including the build that includes projects in private repos). Thanks again!
scabug
commented
9 years ago @gbasler said: Fixed by #8999.
I have some non-trivial projects mostly written using Scala called Sireum (http://sireum.org); the projects compile well with 2.11.2 within ~ 1 minute (Java JDK 8u25, OS X 10.10.1).
However, scalac 2.11.4 seems to have a serious performance issue when compiling Sireum such that I gave up after waiting for 27+ minutes, (the CPU is still at 99-100%).
Here's how one can reproduce the issue:
I tried setting off the pattern matching depth per 2.11.4 compiler warnings, but it didn't help.
Furthermore, the projects above are only from Sireum public repositories, when I compile the full set of projects including the ones from private repositories, I noticed that 2.11.2 compilation consumes at most ~ 2.6 GB (observed through OS X's ActivityMonitor). However, in 2.11.4, the whole compilation was out of memory (exception thrown), even after I increased the max heap to 8 GB in the sbt script).
Robby