Spore header and `capture`: why `capture` only handles identifiers

[jvican]

There are two ways of defining a spore environment variable: either you define it in the spore header or you use capture. Although both look like the same, there is a subtle difference between these two strategies. Defining the variables in the spore header forces the spore body to have references to locally-defined variables, while the second one allows the spore body to have references to the outer scope.

In previous versions of spores, where you could capture a stable path with capture, this lead to problems. One of these problems is serialization. As the compiler rewrites outer variables/members with their fully qualified name, this implies that this rewrite leads to capturing this. Capturing this will cause serialization runtime errors. For instance, the following code snippet:

class ThisReference {
  val v1 = 10
  import scala.spores._
  val s: Spore[Int, Unit] = spore {
    (x: Int) => println("arg: " + x + ", c1: " + capture(v1))
  }
}

gets transformed by the compiler to:

class ThisReference {
  val v1 = 10
  import scala.spores._
  val s: Spore[Int, Unit] = spore {
    (x: Int) => println("arg: " + x + ", c1: " + capture(ThisReference.this.v1))
  }
}

If we execute the following spore in, for instance, Spark, it will fail because this cannot be serialized.

Solving this problem requires:

1. Detecting the use of capture and letting the macro rewrite the "captured" variables as if they had been written in the spore header.
2. Forbidding stable paths in the argument of capture.

As we don't want to change the evaluation semantics of spores, we need to go with 2. This causes some inconveniences in the user because capturing outer members will always lead to compilation errors. There are two workarounds: use the explicit spore definition (defining a spore header) or declare a local variable outside of the spore definition and reference to that one instead.

Solution 1

class ThisReference {
  val v1 = 10
  import scala.spores._
  val s: Spore[Int, Unit] = {
    val forcedLocalVariable = v1
    spore {
      (x: Int) => println("arg: " + x + ", c1: " + capture(forcedLocalVariable))
    }
  }
}

Solution 2

class ThisReference {
  val v1 = 10
  import scala.spores._
  val s: Spore[Int, Unit] = {
    val forcedLocalVariable = v1
    spore {
      (x: Int) => println("arg: " + x + ", c1: " + capture(forcedLocalVariable))
    }
  }
}

When using for-comprehensions, the first solution is not possible. Use workaround 2 instead.