add persistsPartition to support offset update within foreachRDD

dibbhatt / kafka-spark-consumer

High Performance Kafka Connector for Spark Streaming.Supports Multi Topic Fetch, Kafka Security. Reliable offset management in Zookeeper. No Data-loss. No dependency on HDFS and WAL. In-built PID rate controller. Support Message Handler . Offset Lag checker.

Apache License 2.0

635 stars 318 forks source link

add persistsPartition to support offset update within foreachRDD #65

Closed doddys closed 5 years ago

doddys commented 5 years ago

add another alternative to commit offset within foreachRDD instead at the DStream level.

dibbhatt commented 5 years ago

Thanks @doddys . I will do a release today with this changes .