Out of memory while compiling

[Skeebl]

I'm getting java.lang.OutOfMemoryError: Java heap space while compiling my project with pickling since a recent update. It used to compile fine last week, and it still works fine as long as I comment simulation.pickle in class Parser.

I'm running sbt compile with default parameters: /usr/bin/java -Xmx512M -jar /usr/local/Cellar/sbt/0.13.0/libexec/sbt-launch.jar compile but I have tried to increase my heap up to 4GB, only to get: java.lang.OutOfMemoryError: GC overhead limit exceeded

The process uses 100% of one cpu until it reaches the heap limit, then it uses every cpu as it is forced to GC.

Most recent error log Another error log

I'm currently tring to reproduce with a smaller part of the project, but I only manage to get java.io.IOException: File name too long as in #10 instead of the intended OutOfMemoryError.

[heathermiller]

Thanks for taking the time to try scala/pickling and to report this!

Funny story – I can compile gatling-debug, but this totally ridiculous thing happens:

[success] Total time: 481 s, completed Sep 10, 2013 10:09:24 PM

We made some changes yesterday and republished the artifact, will see if those changes help your use case. In any case, we're on it, thanks :)

[Skeebl]

Thanks for your answer.

Indeed, 481 seconds seems a bit excessive to compile a few classes ;) Did you allocate more than 4GB to the heap ?

The new artifact does not seem to resolve the problem. I hope this problem will not delay too much your release.

[heathermiller]

Yep, it was also with a 4GB heap. We're on it though – ideally we'll have this resolved today. Thanks for hanging in there :)

[ephe-meral]

Got a similar problem in a project of mine, involving a (not too overly complex) trait + case class hierarchy. It seems to me that (guessing from the pretty long log-implicits output) the picklers of the classes that are lowest in hierarchy are generated over and over when referenced by the trait they implement in other case classes?

I'm on scala 2.10.2, sbt 0.13.0, pickling 0.8.0-SNAPSHOT. This is the project, with this file in particular - i'm calling pickle on a Message directly in this file. And i'm still not sure if master actually compiles.

Edit This small test is already too much for my pc...: https://gist.github.com/ephe-meral/6715045

[phaller]

Yay!

[success] Total time: 74 s, completed Oct 9, 2013 7:50:53 PM

Pushing a fix soon...

[heathermiller]

Philipp's successful compile was for @Skeebl's gatling project. @ephe-meral, we'll also test it with your entice project and your gist – thanks a lot for sharing them.

...This has been a hell of a bug to fix. It required some pretty significant reworking to how we deal with collections. Thanks to @phaller for his valiant effort!

[phaller]

Fixed by c5e40594ea38c26e235d3e75bd6095e0da59e8df

[Skeebl]

Thanks for you dedication to this problem. I'll test it as soon as the snapshot is updated.