fluent / fluent-bit

Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
https://fluentbit.io
Apache License 2.0
5.88k stars 1.59k forks source link

Allow disabling of tail event logs for tail inputs watching Fluentbit's own logging output file #9641

Open nuclearpidgeon opened 18 hours ago

nuclearpidgeon commented 18 hours ago

If Fluentbit is set up to log out to a file, and then also set up with a tail input on that file, in debug mode, a tonne of events get logged in a loopback manner.

Sample config:

[SERVICE]
    Log_File /tmp/fluentbit.log

[INPUT]
    Name tail
    Path /tmp/fluentbit.log
    Tag ouroborous

[INPUT]
    Name dummy

[OUTPUT]
    Name stdout
    Match *

Sample output:

[2024/11/25 07:26:29] [ info] [sp] stream processor started
[2024/11/25 07:26:29] [ info] [output:stdout:stdout.0] worker #0 started
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] [static files] processed 1.2K
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] inode=5778485 file=/tmp/fluentbit.log promote to TAIL_EVENT
[2024/11/25 07:26:29] [ info] [input:tail:tail.0] inotify_fs_add(): inode=5778485 watch_fd=1 name=/tmp/fluentbit.log
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] [static files] processed 0b, done
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] inode=5778485, /tmp/fluentbit.log, events: IN_MODIFY 
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] inode=5778485, /tmp/fluentbit.log, events: IN_MODIFY 
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] inode=5778485, /tmp/fluentbit.log, events: IN_MODIFY 
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] inode=5778485, /tmp/fluentbit.log, events: IN_MODIFY 
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] inode=5778485, /tmp/fluentbit.log, events: IN_MODIFY 
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] inode=5778485, /tmp/fluentbit.log, events: IN_MODIFY 
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] inode=5778485, /tmp/fluentbit.log, events: IN_MODIFY 
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] inode=5778485, /tmp/fluentbit.log, events: IN_MODIFY 
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] inode=5778485, /tmp/fluentbit.log, events: IN_MODIFY 
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] inode=5778485, /tmp/fluentbit.log, events: IN_MODIFY 
[2024/11/25 07:26:29] [debug] [input:tail:tail.0] inode=5778485, /tmp/fluentbit.log, events: IN_MODIFY 
...etc...

It'd be great if this tail event debug logging were selectively disabled for any tail inputs that happen to be targeting the same file that Fluentbit itself is writing logs out to, so that debug logging mode was still able to be used for useful debugging when loopback-logging like this is in place.