scalameta pickling

[MasseGuillaume]

given https://github.com/MasseGuillaume/cats/commit/b5d25c3cd99b05b59df3ea0477d800319051285d

when sbt ";core/compile;metadocCompilerPlugin/console"

pickling fails with: fatal error: unpicklee cache is corrupted at 7

[eed3si9n]

The error message is coming from https://github.com/scala/pickling/blob/v0.10.1_2.11/core/src/main/scala/scala/pickling/internal/package.scala#L152-L161 likely some issue in structure sharing.

[phaller]

@eed3si9n is right, it's an issue with structure sharing. You can try to disable structure sharing:

import scala.pickling.shareNothing._

[phaller]

For the next pickling release we should get scalameta pickling working.

[MasseGuillaume]

@phaller If i disable structure sharing I get a stack overflow. I think this is because in the AST a node has a reference to his parent and vice versa. So there is a loop.

[phaller]

@MasseGuillaume I see. Then, indeed, structure sharing is required. It would be helpful to have a smaller example that exposes this bug, though.

[MasseGuillaume]

@phaller there you go: https://github.com/MasseGuillaume/scalameta-pickling-issue

[phaller]

@MasseGuillaume Great, thanks! I'll have a look at it.

[phaller]

@MasseGuillaume I'm running into issues building the meta project (plugin builds fine):

[error] 669 errors were encountered during merge
[trace] Stack trace suppressed: run last plugin/*:assembly for the full output.
[error] (plugin/*:assembly) deduplicate: different file contents found in the following:
[error] /Users/phaller/.ivy2/cache/org.scalameta/scalahost_2.11.7/jars/scalahost_2.11.7-0.1.0-SNAPSHOT.jar:org/scalameta/adt/AdtMacros$$anon$1.class
[error] /Users/phaller/.ivy2/cache/org.scalameta/foundation_2.11/jars/foundation_2.11-0.1.0-SNAPSHOT.jar:org/scalameta/adt/AdtMacros$$anon$1.class
[error] deduplicate: different file contents found in the following:
[error] /Users/phaller/.ivy2/cache/org.scalameta/scalahost_2.11.7/jars/scalahost_2.11.7-0.1.0-SNAPSHOT.jar:org/scalameta/adt/AdtMacros$$anon$2.class
[error] /Users/phaller/.ivy2/cache/org.scalameta/foundation_2.11/jars/foundation_2.11-0.1.0-SNAPSHOT.jar:org/scalameta/adt/AdtMacros$$anon$2.class
...

Any hints how to resolve this?

[MasseGuillaume]

@xeno-by ping @phaller delete your ivy cache rm -rf ~/.ivy2/cache/org.scalameta/ and publishLocal scalameta projects in the following order:

1. scalameta
2. interpreter
3. scalahost

[phaller]

OK, perfect, thanks!

[xeno-by]

@MasseGuillaume should work now, but I really have no clue how to fix it in the long run unless sbt changes the way they handle compiler plugins.

[phaller]

@MasseGuillaume I can build and run the compiler plugin now, but I couldn't reproduce the bug, yet, unfortunately. I'll keep looking.

[MasseGuillaume]

@phaller if you need help we can do a Google Hangout. You can send and chat invite to masgui@gmail.com.

[phaller]

@MasseGuillaume OK, sounds good!

[MasseGuillaume]

meta/test

[MasseGuillaume]

@phaller any updates ?

[phaller]

@MasseGuillaume sorry, I had to context-switch to other things. However, I found out that it was trying to use the runtime pickler/unpickler which caused issues. I think we need to try to have picklers generated systematically for all involved classes. If picklers cannot be generated via the Pickler.generate macro, try custom picklers.

[jsuereth]

@MasseGuillaume I just finsihed a custom pickler tutorial, if you need: https://github.com/scala/pickling/wiki/Writing-custom-Picklers-Unpicklers

Let me know if you need any help getting scala-meta compiled using picklers.

[MasseGuillaume]

@jsuereth thanks I will try this approach ;-)