SeisoLLC / zeek-kafka

A Zeek log writer plugin that publishes to Kafka.
Apache License 2.0
45 stars 15 forks source link

How to add Partition in local.zeek configuration #55

Open cybersecurity99 opened 2 years ago

cybersecurity99 commented 2 years ago

Summary of the issue

I want to add Partition in local.zeek file that contain configuration for kafka.

Expected behavior

To create a x number of partition in all files

Steps to reproduce

...

Where applicable, consider providing a patch that uses the end to end testing environment.

Logs, errors, etc.

I am trying ["partition"]=10 in Kafka::Config But it is not working

Your environment

ottobackwards commented 2 years ago

the plugin does not do per topic configuration, and I think the librdkafka does partition strategy configuration in the topic configuration not the global. All the settings in the kafka configuration are global.

To support this we would have to support new configuration local -> topic creation.

At least that is my read. @JonZeolla ?

ottobackwards commented 2 years ago

@cybersecurity99 if you look at librdkaka and see how you would want to configuration based on what it supports, can you give us an example? https://docs.confluent.io/5.5.0/clients/librdkafka/md_CONFIGURATION.html