confluentinc / librdkafka

The Apache Kafka C/C++ library
Other
175 stars 3.14k forks source link

How to specify partition number when create a topic #166

Closed majic31 closed 9 years ago

majic31 commented 9 years ago

Librdkafka provides RdKafka::Topic::create method to create a topic. But I can't specify partition number. I try to find struct of rd_kakfa_properties in Rdkafka_defaultconf.c, but failed to find related config. Please help me~~~ Thanks~

edenhill commented 9 years ago

The partition is not instantiated as an object but is simply provided as an integer to the ::produce() function, where it may either be an absolute partition (0..N) or RdKafka::Topic::PARTITION_UA for using the configured partitioner.

From rdkafkacpp.h:

   * 'partition' is the target partition, either:
   *   - RdKafka::Topic::PARTITION_UA (unassigned) for
   *     automatic partitioning using the topic's partitioner function, or
   *   - a fixed partition (0..N)
majic31 commented 9 years ago

Thanks for you answer. I means one topic including one or more partitions. I can create topic, which includes default partition number("num.partitions" in server.properties kafka configure file). But I can't assign number of partitions when create topic. Does librdkafka provide related interface? Thank you :) Best regards:)

edenhill commented 9 years ago

Specifying the number of partitions for a topic is a broker side thing, the clients, librdkafka included, reads the number of partitions created from the broker, so you dont need to do any configuration of available partitions on the client side (librdkafka). Does that answer your question?

majic31 commented 9 years ago

Ps: I can create topic using RdKafka::Topic::create() method. The number of partitions in the topic is equal to "num.partitions" in server.properties configure file.

edenhill commented 9 years ago

Ah, yes, so if 'auto.create.topics.enable=true' is configured on the broker any unknown topic requested by the client will be automatically created using the default parameters in server.properties. There is no way for the client to specify the number of partitions in this case, it is a broker side thing.

majic31 commented 9 years ago

Ok. I get it. Thank you ! :)