Make it possible seeing REPL definitions from toolbox parser

[scabug]

When using the toolbox parser from REPL, one cannot get parsed code to see REPL definitions because of the way they are wrapped on objects.

It would be useful to be able to get around that.

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-7081?orig=1 Reporter: @dcsobral Affected Versions: 2.10.0

[scabug]

@xeno-by said (edited on Feb 15, 2013 1:21:07 PM UTC): In the meanwhile you can use this workaround to get to the list of symbols defined in repl and to import them explicitly in your toolbox:

scala> val replClassloader = Thread.currentThread().getContextClassLoader
replClassloader: ClassLoader = scala.tools.nsc.interpreter.IMain$TranslatingClassLoader@197b59e2

scala> val outerField = replClassloader.getClass.getFields.head
outerField: java.lang.reflect.Field = public final scala.tools.nsc.interpreter.IMain scala.tools.nsc.interpreter.IMain$TranslatingClassLoader.$outer

scala> val interpreter = outerField.get(replClassloader).asInstanceOf[scala.tools.nsc.interpreter.IMain]
interpreter: scala.tools.nsc.interpreter.IMain = scala.tools.nsc.interpreter.ILoop$ILoopInterpreter@7c4e758a

scala> interpreter.definedSymbolList
res1: List[interpreter.memberHandlers.intp.global.Symbol] = List(value $intp, value cm, value rm, value tb, method mirrorThatLoaded, class iw$SymbolFlags, class iw$TypeFlags, value replClassloader, value outerField, value interpreter)

[scabug]

@dcsobral said: How do I import them into toolbox?

[scabug]

@xeno-by said: By injecting s"import ${symbol.fullName}" lines into the code being compiled.

[scabug]

@dcsobral said: I tried fullName and it didn't add the full path. Then I tried fullNameString and that did add a path, but it still couldn't import them. Maybe my version is too old? It's post 2.10.0, at any rate.

[scabug]

@xeno-by said: Hmm that's not good. Could you share the code?

[scabug]

@dcsobral said: Not any more code than what's above, really:

scala> val x = 5
x: Int = 5

scala> val replClassloader = Thread.currentThread().getContextClassLoader
replClassloader: ClassLoader = scala.tools.nsc.interpreter.IMain$TranslatingClassLoader@ab245dc

scala> val outerField = replClassloader.getClass.getFields.head
outerField: java.lang.reflect.Field = public final scala.tools.nsc.interpreter.IMain scala.tools.nsc.interpreter.IMain$TranslatingClassLoader.$outer

scala> val interpreter = outerField.get(replClassloader).asInstanceOf[scala.tools.nsc.interpreter.IMain]
interpreter: scala.tools.nsc.interpreter.IMain = scala.tools.nsc.interpreter.ILoop$ILoopInterpreter@46993aaa

scala> interpreter.definedSymbolList
res0: List[interpreter.memberHandlers.intp.global.Symbol] = List(value $intp, value x, value replClassloader, value outerField, value interpreter)

scala> import scala.tools.reflect.ToolBox
ToolBox          ToolBoxError     ToolBoxFactory

scala> import scala.tools.reflect.ToolBox
import scala.tools.reflect.ToolBox

scala> import scala.reflect.runtime.{currentMirror => cm}
import scala.reflect.runtime.{currentMirror=>cm}

scala> val tb = cm.mkToolBox()
tb: scala.tools.reflect.ToolBox[reflect.runtime.universe.type] = scala.tools.reflect.ToolBoxFactory$ToolBoxImpl@b6a6597

scala> val imports = res0.map(i => s"import ${i.fullNameString}").mkString("", "; ", "; ")
imports: String = "import $line2.iw.$intp; import $line3.iw.x; import $line4.iw.replClassloader; import $line5.iw.outerField; import $line6.iw.interpreter; "

scala> tb.eval(tb.parse(imports+"x"))
scala.tools.reflect.ToolBoxError: reflective compilation has failed:

object $intp is not a member of package $line2.iw
object x is not a member of package $line3.iw
object replClassloader is not a member of package $line4.iw
object outerField is not a member of package $line5.iw
object interpreter is not a member of package $line6.iw
not found: value x
        at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.throwIfErrors(ToolBoxFactory.scala:314)
        at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.compile(ToolBoxFactory.scala:250)
        at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl.compile(ToolBoxFactory.scala:409)
        at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl.eval(ToolBoxFactory.scala:412)
        at .<init>(<console>:16)
        at .<clinit>(<console>)
        at .<init>(<console>:7)
        at .<clinit>(<console>)
        at $print(<console>)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:689)
        at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:895)
        at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:543)
        at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:574)
        at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:538)
        at scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:567)
        at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:612)
        at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:524)
        at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:380)
        at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:397)
        at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:680)
        at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:666)
        at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:666)
        at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:95)
        at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:666)
        at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:81)
        at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:94)
        at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)
        at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)

scala>

[scabug]

@xeno-by said: That's doubly weird. Here's what I get for Scala 2.10.0 downloaded from the official site (so that all the possibility of my environment inducing non-obvious errors is excluded):

09:57 ~/Downloads$ cd scala-2.10.0
09:57 ~/Downloads/scala-2.10.0$ bin/scala -Xprint:parser
Welcome to Scala version 2.10.0 (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_37).
Type in expressions to have them evaluated.
Type :help for more information.

<snip>

scala> val x = 2
[[syntax trees at end of                    parser]] // <console>
package $line3 {
  object $read extends scala.AnyRef {
    def <init>() = {
      super.<init>();
      ()
    };
    object $iw extends scala.AnyRef {
      def <init>() = {
        super.<init>();
        ()
      };
      object $iw extends scala.AnyRef {
        def <init>() = {
          super.<init>();
          ()
        };
        val x = 2
      }
    }
  }
}

[[syntax trees at end of                    parser]] // <console>
package $line3 {
  object $eval extends scala.AnyRef {
    def <init>() = {
      super.<init>();
      ()
    };
    lazy val $result = $line3.$read.$iw.$iw.x;
    val $print: String = {
      $read.$iw.$iw;
      "".$plus("x: Int = ").$plus(scala.runtime.ScalaRunTime.replStringOf($line3.$read.$iw.$iw.x, 1000))
    }
  }
}

x: Int = 2

scala> val y = x
[[syntax trees at end of                    parser]] // <console>
package $line4 {
  object $read extends scala.AnyRef {
    def <init>() = {
      super.<init>();
      ()
    };
    object $iw extends scala.AnyRef {
      def <init>() = {
        super.<init>();
        ()
      };
      import $line3.$read.$iw.$iw.x;
      object $iw extends scala.AnyRef {
        def <init>() = {
          super.<init>();
          ()
        };
        val y = x
      }
    }
  }
}

[[syntax trees at end of                    parser]] // <console>
package $line4 {
  object $eval extends scala.AnyRef {
    def <init>() = {
      super.<init>();
      ()
    };
    lazy val $result = $line4.$read.$iw.$iw.y;
    val $print: String = {
      $read.$iw.$iw;
      "".$plus("y: Int = ").$plus(scala.runtime.ScalaRunTime.replStringOf($line4.$read.$iw.$iw.y, 1000))
    }
  }
}

y: Int = 2

Here we see that x gets translated to $line3.$read.$iw.$iw.x, which is by far not the owner chain reported by fullNameString. That's strange.

[scabug]

@xeno-by said: There was a phase mismatch problem. You can't just do sym.fullName, you need to do afterPhase(typerPhase)(sym.name):

scala> interpreter.definedSymbolList foreach (sym => println(interpreter.global.afterPhase(interpreter.global.currentRun.typerPhase)(sym.fullName)))
$line2.$read.$iw.$iw.$intp
$line3.$read.$iw.$iw.cm
$line3.$read.$iw.$iw.rm
$line3.$read.$iw.$iw.tb
$line3.$read.$iw.$iw.mirrorThatLoaded
$line3.$read.$iw.$iw.SymbolFlags
$line3.$read.$iw.$iw.TypeFlags
$line4.$read.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.replClassloader
$line8.$read.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.outerField
$line9.$read.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.interpreter
$line10.$read.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.res0
$line11.$read.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.res1
$line13.$read.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.$iw.res3

[scabug]

@xeno-by said: Btw when trying out my code beware of the WAT?!: http://groups.google.com/group/scala-internals/browse_thread/thread/c1d1815ae3f6378e

[scabug]

@dcsobral said (edited on Feb 24, 2013 5:01:18 AM UTC): What's the cast you did on interpreter to get global.afterPhase? Here it says that:

scala> interpreter.definedSymbolList foreach (sym => println(interpreter.global.afterPhase(interpreter.global.currentRun.typerPhase)(sym.fullName)))
<console>:15: error: value afterPhase is not a member of scala.tools.nsc.Global
              interpreter.definedSymbolList foreach (sym => println(interpreter.global.afterPhase(interpreter.global.currentRun.typerPhase)(sym.fullName)))
                                                                                      ^

[scabug]

@xeno-by said (edited on Feb 24, 2013 9:51:35 AM UTC): Strange. This shouldn't require any casts as in https://github.com/scala/scala/blob/v2.10.0/src/reflect/scala/reflect/internal/SymbolTable.scala#L210. What version of scala do you use?

[scabug]

@dcsobral said: 76f167, I think. A master from February 13th.

[scabug]

@som-snytt said: Use exitingPhase now, apparently.