Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.24k stars 1.05k forks source link

improve journal metrics and retention #2368

Open jalogisch opened 8 years ago

jalogisch commented 8 years ago

if you stop processing in graylog and the journal get filled and will drop the oldest message by time or if space limit is reached.

# Journal hold messages before they could be written to Elasticsearch.
# For a maximum of 12 hours or 5 GB whichever happens first.
# During normal operation the journal will be smaller.
#message_journal_max_age = 12h
message_journal_max_size = 1gb

But the statistics of the node, means the message count will not show the drop of the messages. Only if you proceed your processing the message count will suddenly drop to the real value.

ideas

kroepke commented 7 years ago

This information is currently not directly accessible in the journal implementation we are using, so this needs to be moved out.