search

fluent-plugins-nursery / fluent-plugin-systemd

This is a fluentd input plugin. It reads logs from the systemd journal.
Apache License 2.0
153 stars 43 forks source link

pause reading from journal when there is back pressure #37

Closed richm closed 7 years ago

richm commented 7 years ago

If the buffer queue is full, router.emit by default will throw an exception BufferOverflowError https://github.com/fluent/fluentd/blob/master/lib/fluent/plugin/buffer.rb#L196 when this happens, the journal plugin will just throw away the record: https://github.com/reevoo/fluent-plugin-systemd/blob/master/lib/fluent/plugin/in_systemd.rb#L95 Instead, the plugin should back off and allow the queue to drain, as is done by e.g. the in_tail plugin. Right now, you can get this same sort of behavior by using buffer_queue_full_action block, but you have to configure every output plugin with this. It would be better if the in_systemd plugin itself did the flow control.

errm commented 7 years ago

Thanks @richm, I will look at a fix for this...