CSIRT-MU / Stream4Flow

A framework for the real-time network traffic analysis based on world-leading technologies for distributed stream processing, network traffic monitoring, and visualization.
https://csirt.muni.cz/?lang=en
MIT License
101 stars 36 forks source link

Template of kafka server.settings to reduce log size #83

Closed tomjirsa closed 6 years ago

tomjirsa commented 6 years ago

Add template for kafka config file server.settings with following code:

############################# Log Retention Policy #############################

# The following configurations control the disposal of log segments. The policy can
# be set to delete segments after a period of time, or after a given size has accumulated.
# A segment will be deleted whenever *either* of these criteria are met. Deletion always happens
# from the end of the log.

# The minimum age of a log file to be eligible for deletion
log.retention.hours=0
log.retention.minutes=10

# A size-based retention policy for logs. Segments are pruned from the log as long as the remaining
# segments don't drop below log.retention.bytes.
#log.retention.bytes=1073741824

# The maximum size of a log segment file. When this size is reached a new log segment will be created.
log.segment.bytes=1073741824

# The interval at which log segments are checked to see if they can be deleted according
# to the retention policies
log.retention.check.interval.ms=300000

# Allow topic delete
delete.topic.enable=true
tomjirsa commented 6 years ago

working on solution in branch #83-kafka-template

tomjirsa commented 6 years ago

Finished, merged in #86, closing.