Polish support for checkpointing

[juanrh]

That should include, at least:

• hygienize closures in DStreamProp.forall. Currently currFormulaLock is defined as new Serializable{} instead of new Object to avoid a serialization exception during checkpointing. Should use the techniques in http://erikerlandson.github.io/blog/2015/03/31/hygienic-closures-for-scala-function-serialization/ to replace that hack, note we also hygienized closures in https://github.com/juanrh/sscheck/blob/91ff63d5c0dd0007939dd588fb8c49d852f1d097/src/main/scala/es/ucm/fdi/sscheck/matcher/specs2/package.scala, used in https://github.com/juanrh/sscheck/blob/b18347613487a2e3b2479e2ad011ae8f88725838/src/test/scala/es/ucm/fdi/sscheck/spark/streaming/SharedStreamingContextBeforeAfterEachTest.scala
• stopping the streaming context leads to messages

15/09/23 16:59:21 WARN SharedStreamingContext: stopping test Spark Streaming context
15/09/23 16:59:21 ERROR CheckpointWriter: Could not submit checkpoint task to the thread pool executor
java.util.concurrent.RejectedExecutionException: Task org.apache.spark.streaming.CheckpointWriter$CheckpointWriteHandler@56e81962 rejected from java.util.concurrent.ThreadPoolExecutor@1fda9d81[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 61]
    at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
    at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
    at org.apache.spark.streaming.CheckpointWriter.write(Checkpoint.scala:244)
    at org.apache.spark.streaming.scheduler.JobGenerator.doCheckpoint(JobGenerator.scala:291)
    at org.apache.spark.streaming.scheduler.JobGenerator.org$apache$spark$streaming$scheduler$JobGenerator$$processEvent(JobGenerator.scala:180)
    at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:83)
    at org.apache.spark.streaming.scheduler.JobGenerator$$anon$1.onReceive(JobGenerator.scala:82)
    at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
15/09/23 16:59:21 WARN SharedSparkContext: stopping test Spark context

this doesn't break the tests but it is not nice. We could try first stopping gracefully and only then stopping directly, and also try our luck in the spark mailing list

[juanrh]

Consider using the following code to stop the streaming context in DStreamProp.forall when a counterexample is found

Try { 
            ssc.stop(stopSparkContext = false, stopGracefully = true) 
          } recover {
            case _ => ssc.stop(stopSparkContext=false, stopGracefully=false)
          }

[juanrh]

fixed in 0.2.1-SNAPSHOT when using the trait DStreamTLProperty and calling `forAllDStream