jaceklaskowski / spark-activator

Spark Streaming with Scala and Akka Activator template
https://typesafe.com/activator/template/spark-streaming-scala-akka
Apache License 2.0
44 stars 26 forks source link

Spark Streaming with Scala and Akka activator template

Build Status Stories in Ready

This is an Typesafe Activator template to demonstrate Apache Spark for near-real-time data stream processing using Scala and Akka using the Spark Streaming extension.

The motivation was Go Reactive Activator Contest: Scala Days Edition

Run the activator project

Once you download the activator template, execute ./activator clean run.

When the words Hello from Spark Streaming with Scala and Akka arrive to the actor and Spark Streaming gets notified about them (by the store method), the flow finishes and so does the demo.

Watch out for the following message in the logs (the Time line may show a different value):

-------------------------------------------
Time: 1422714451000 ms
-------------------------------------------
Hello from Spark Streaming with Scala and Akka

Please note that the clean task is important after the first run as some files get serialized and then deserialized that have often caused deserialization problems. As a workaround, running clean gets rid of the potential issue quickly and easily.