search

SeisoLLC / zeek-kafka

A Zeek log writer plugin that publishes to Kafka.
Apache License 2.0
45 stars 16 forks source link

How to configure Bucket size , message size and message per second rate #42

Open cybersecurity99 opened 2 years ago

cybersecurity99 commented 2 years ago

Summary of the issue

... I want to measure the rate at which Kafka will be processing for 1G or 10G /s rate So How to configure Bucket size , message size and message per second rate to check the sizing of kafka in this plugin anyway to modify or check that

Your environment

cybersecurity99 commented 2 years ago

Hi @JonZeolla any view on this

JonZeolla commented 2 years ago

@cybersecurity99 I'm not sure we have a great answer for this. You can configure anything that the underlying library accepts via kafka_conf (documented in https://github.com/SeisoLLC/zeek-kafka#kafka_conf) so there may be something useful downstream from us.

Another thing you could do if you wanted to gather information about what's currently happening is configure certain debug logs. These are also passed through to the underlying library, and are documented in https://github.com/SeisoLLC/zeek-kafka#debug

JonZeolla commented 2 years ago

@cybersecurity99 did you have any additional questions on this one?