search

Stratio / Decision

Powered by Spark Streaming & Siddhi
https://stratio.atlassian.net/wiki/display/DECISION0x9/Home
Apache License 2.0
315 stars 89 forks source link

Use the Direct Approach in Streaming-Kafka integration. #34

Open mariomgal opened 9 years ago

mariomgal commented 9 years ago

Spark 1.3 introduced a new direct approach for integrating Kafka and Spark Streaming (without receivers). It has several advantages, as explained here:

https://spark.apache.org/docs/1.3.1/streaming-kafka-integration.html

mariomgal commented 9 years ago

Take High Availability features into account, we will need to store Kafka status in Zookeeper.