Closed snowch closed 6 years ago
Spark repository provides kafka examples. Perhaps these could be adapted for messagehub and added to this repository?
https://github.com/apache/spark/blob/branch-1.6/examples/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala https://github.com/apache/spark/blob/branch-2.0/examples/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala https://github.com/apache/spark/blob/branch-2.1/examples/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala
+1 pyspark equivalent would be very useful!
+1
@snowch Sorry for the extremely late answer. Not sure our samples repository would be a good place for that but I think this could be a great Code Pattern (https://developer.ibm.com/patterns/). Happy to help you if you want to build it.
It would be good to see some examples of connecting to MessageHub via spark streaming using both python and scala.