search

rsyslog / rsyslog-doc

documentation for the rsyslog project
Other
98 stars 250 forks source link

High/low watermark levels to discard logs #976

Open Ashlanfox opened 2 years ago

Ashlanfox commented 2 years ago

Reading the documentation I thought high/low watermark levels could be used to discard logs (see https://www.rsyslog.com/doc/master/whitepapers/reliable_logging.html):

You can configure rsyslog to have high/low watermark levels, when the queue fills past the high watermark, rsyslog will start discarding logs below a specified severity, and stop doing so when it drops below the low watermark level

However this conflicts with rsyslog queue parameters (see https://www.rsyslog.com/doc/master/rainerscript/queue_parameters.html):

queue.highWatermark This applies to disk-assisted queues, only. When the queue fills up to this number of messages, the queue begins to spool messages to disk. Please note that this should not happen as part of usual processing, because disk queue mode is very considerably slower than in-memory queue mode. Going to disk should be reserved for cases where an output action destination is offline for some period.

Is one part of the documentation faulty or did I miss something ?

davidelang commented 1 year ago

there are two types of watermarks

  1. the level at which it will start putting messages in the disk queue instead of memory

  2. the level at which it will start throwing away messages (with modifiers to limit severity that is thrown away)

you can configure either of both of these, and it's up to you to set the level to spill to disk to be below the level at which you are throwing away messages (it's still possible for that to be valid with severity limits)

this is very advanced level stuff.

David Lang

On Thu, 28 Jul 2022, Ashlanfox wrote:

Reading the documentation I thought high/low watermark levels could be used to discard logs (see https://www.rsyslog.com/doc/master/whitepapers/reliable_logging.html):

You can configure rsyslog to have high/low watermark levels, when the queue fills past the high watermark, rsyslog will start discarding logs below a specified severity, and stop doing so when it drops below the low watermark level

However this conflicts with rsyslog queue parameters (see https://www.rsyslog.com/doc/master/rainerscript/queue_parameters.html):

queue.highWatermark This applies to disk-assisted queues, only. When the queue fills up to this number of messages, the queue begins to spool messages to disk. Please note that this should not happen as part of usual processing, because disk queue mode is very considerably slower than in-memory queue mode. Going to disk should be reserved for cases where an output action destination is offline for some period.

Is one of the documentation faulty or did I miss something ?