Allow fluentbit exec plugin to process forevergoing commands like mpstat

[LukasJerabek]

Is your feature request related to a problem? Please describe. I would like to be able to parse outputs of these commands with exec plugin: iostat -d 1 mpstat 1 pidstat 1 These are neverending commands but fluentbit expects the command to finish. However for example fluentd manages this and creates record from every line these commands output at the time they output it - continually.

Describe the solution you'd like In fluentd when you dont include the alternative of fluentbits "Interval_Sec" it runs once and it does not matter if the command finishes or not, it processes all the lines the command outputs continually. In fluentbit there is already the "Oneshot" parametr. I am not sure about its behaviour, in documentations it says: "This allows collection of data precedent to fluent-bit's startup (bool, default: false)" - I am dont get how could I collect data precedent to fluentd-bits startup with that. I guess that if you use Oneshot parametr you dont have to use Interval_Sec parametr and command goes once? Since this is unclear for me, its hard for me to suggest solution. The best would be if the fluentbit could process the lines that the command outputs, no matter if the command finishes or not or what parameters are set, out of the box.

Describe alternatives you've considered

• None came to my mind, maybe there are some, but continuous output commands exist and if not for these commands the need might arise for some other command.

Additional context I am trying be flexible in what I monitor and exec almost allows that. Also using exec is easy because you can use any bash command without the need to learn how to do go custom plugins.

I was trying this config and it outputs only after I kill the fluentbit process (you need to have sysstat package installed, its comonly available in apt, dnf,...)

path/to/fluent-bit -c fluent-bit.conf

[SERVICE]
    Flush        5
    Daemon       Off
    Log_Level    info
    parsers_file /home/vagrant/fluentbit_parsers.conf
    storage.path /home/vagrant/storage

[INPUT]
    Name          exec
    tag           psql_statements
    command       pidstat -dHhurv 1 | grep  --line-buffered -E '^[0-9]' | stdbuf -oL -eL tr -s " \t" " "
    Oneshot       true
    storage.type  filesystem
    Buf_Size      500mb
    mem_buf_limit 30mb

[OUTPUT]
    Name          stdout
    Match         *

[patrick-stephens]

There is actually a PR on this somewhere...

EDIT: https://github.com/fluent/fluent-bit/pull/6102

[LukasJerabek]

Seems little bit stuck after the suggestion to extend the in_exec plugin instead.

[patrick-stephens]

I suspect just busy with other things but let's see, or feel free to take up the baton too.

[LukasJerabek]

Sure that can be. I cant promise that right now...

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[LukasJerabek]

Still relevant

[leonardo-albertovich]

The current approach for this would be running the binary using the exec input plugin in threaded mode which will cause the input plugin to run in background while ingesting without blocking the rest of the system.

[LukasJerabek]

Thank you for your suggestion. Could you please write an example? I am not sure of what threaded mode means.

[edwardpeek-crown-public]

[INPUT]
    name                exec
    tag                 exec.date

    threaded            true
    command             while sleep 1; do date; done

I can confirm that config as above will continually produce and flush output. Service stop signals are not passed through however which makes fluent-bit hang when requested to exit.

Edit: with systemd control group killing and exit_after_oneshot true configured the behaviour is mostly as expected, but still with quirks.

[LukasJerabek]

Looks like after a while running mpstat with threaded true and exit_after_oneshot true mpstat processes keeps stacking up and hang after shutting fluent-bit down with INT and HUP. Is there some other way we should shutdown fluentbit with? @edwardpeek-crown do you observe same problem? Seems we might have to go back to system service for these evergoing commands.

[edwardpeek-crown-public]

after shutting fluent-bit down with INT and HUP. Is there some other way we should shutdown fluentbit with? @edwardpeek-crown do you observe same problem? Seems we might have to go back to system service for these evergoing commands.

We run fluent-bit under systemd which kills all child processes after the main process exits from the SIGTERM.