search

haskell-works / hw-kafka-client

Kafka client for Haskell, including auto-rebalancing consumers
MIT License
139 stars 50 forks source link

Producer: Topic Partitions Size #187

Closed fabfianda closed 2 years ago

fabfianda commented 2 years ago

Hi guys, thanks for the useful lib! I can't find a way to set the topic partitions size from the producer or anywhere else. What am I missing? Thanks

AlexeyRaga commented 2 years ago

@fabfianda Hi, I am not sure what "topic partition size" would mean this context. Could you please elaborate?

fabfianda commented 2 years ago

Sure, see e.g how with stock kafka script you can define the number of partitions a topic should use:

./bin/kafka-topics.sh --create --topic test --replication-factor 1 --partitions 3

Any way to replicate this with the lib?

fabfianda commented 2 years ago

Hi, did I clarify?

AlexeyRaga commented 2 years ago

@fabfianda producers and consumers don't create topics. They just use them. On your broker you can enable auto-creation and specify the default number of partitions, and when you produce a message for a non-existing topic Kafka will create it accordingly. But that's about it. hw-kafka-client doesn't have administrative functions.

Saying that, there is admin-client branch in which we attempted to add admin capabilities, here: https://github.com/haskell-works/hw-kafka-client/blob/admin-client/src/Kafka/Admin.hs

But this work has not been finished. I'll be happy to see PRs and some movements in this area though.

fabfianda commented 2 years ago

Thank you!