ibm-messaging / event-streams-samples

Apache License 2.0
95 stars 168 forks source link

Feature request: spark streaming python+scala example #11

Closed snowch closed 6 years ago

snowch commented 8 years ago

It would be good to see some examples of connecting to MessageHub via spark streaming using both python and scala.

snowch commented 7 years ago

Spark repository provides kafka examples. Perhaps these could be adapted for messagehub and added to this repository?

https://github.com/apache/spark/blob/branch-1.6/examples/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala https://github.com/apache/spark/blob/branch-2.0/examples/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala https://github.com/apache/spark/blob/branch-2.1/examples/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala

owlz84 commented 7 years ago

+1 pyspark equivalent would be very useful!

allisonjberman commented 7 years ago

+1

mimaison commented 6 years ago

@snowch Sorry for the extremely late answer. Not sure our samples repository would be a good place for that but I think this could be a great Code Pattern (https://developer.ibm.com/patterns/). Happy to help you if you want to build it.