search

dibbhatt / kafka-spark-consumer

High Performance Kafka Connector for Spark Streaming.Supports Multi Topic Fetch, Kafka Security. Reliable offset management in Zookeeper. No Data-loss. No dependency on HDFS and WAL. In-built PID rate controller. Support Message Handler . Offset Lag checker.
Apache License 2.0
635 stars 318 forks source link

Add an optional property to specify the number of partitions #9

Closed jedisct1 closed 9 years ago

jedisct1 commented 9 years ago

This adds a new, optional, property named kafka.partitions.number in order to specify the number of partitions for the given topic.

Automatic detection of the number of partitions is great, but is not an option when the driver is not allowed to connect to the Zookeeper servers.

The previous behavior remains unchanged when this property is not given.

dibbhatt commented 9 years ago

Thank you very much for your contribution. Dib 

 On Wednesday, 1 April 2015 3:27 PM, Frank Denis <notifications@github.com> wrote:

This adds a new, optional, property named kafka.partitions.number in order to specify the number of partitions for the given topic.Automatic detection of the number of partitions is great, but is not an option when the driver is not allowed to connect to the Zookeeper servers.The previous behavior remains unchanged when this property is not given. You can view, comment on, or merge this pull request online at:   https://github.com/dibbhatt/kafka-spark-consumer/pull/9 Commit Summary

File Changes

Patch Links:

jedisct1 commented 9 years ago

Wow, that was fast! Thanks!