Use safe storage configuration in example

[ksauzz]

Hi, this is a suggestion to change a config example in README.

Summary

persistent true setting could cause unexpected high IO utilization when systemd-journal already stores much logs. So I think persistent false would be better in example.

Details

I observed that IO utilization continually indicated 90~100% after starting fluentd with fluent-plugin-systemd on a server which had a few GB logs in systemd-journal. Using systemtap I observed a massive number of reads and writes to the position file. After applying persistent false, high IO utilization disappeared.

In my understanding, with persistent mode (persistent true), PosWriter writes a journal position(cursor) to the position file via LocalStorage after every reading an entry from journald. This could be likely to cause the issue I faced. Without persistent mode, a journal position is only stored in memory, but it's periodically flushed to the position file per about 1 sec. So 'persistent false' would be safe to most users as default.

Of cause, I think this problem doesn't happen if journald has not much logs or using read_from_head false.

library versions

• fluentd-plugin-systemd 0.3.1
• fluentd 0.14.20

My configuration of fluent-plugin-systemd

<source>
  @type systemd
  path /var/log/journal
  filters []
  <storage>
    @type local
    persistent true
    path /var/log/td-agent/systemd.pos
  </storage>
  <entry>
    fields_lowercase true
    fields_strip_underscores true
  </entry>
  tag log.systemd
  read_from_head true
</source>

output of systemtap using iotime.stp

1778786 31724 (thread.rb:70) iotime /var/log/td-agent/systemd.pos/worker0/storage.json.tmp time: 22
1787031 31724 (thread.rb:70) access /var/log/td-agent/systemd.pos/worker0/storage.json read: 138 write: 0
1787039 31724 (thread.rb:70) iotime /var/log/td-agent/systemd.pos/worker0/storage.json time: 11
1787186 31724 (thread.rb:70) access /var/log/td-agent/systemd.pos/worker0/storage.json.tmp read: 0 write: 138
1787191 31724 (thread.rb:70) iotime /var/log/td-agent/systemd.pos/worker0/storage.json.tmp time: 24
1795409 31724 (thread.rb:70) access /var/log/td-agent/systemd.pos/worker0/storage.json read: 138 write: 0
1795416 31724 (thread.rb:70) iotime /var/log/td-agent/systemd.pos/worker0/storage.json time: 10
1795554 31724 (thread.rb:70) access /var/log/td-agent/systemd.pos/worker0/storage.json.tmp read: 0 write: 138
1795559 31724 (thread.rb:70) iotime /var/log/td-agent/systemd.pos/worker0/storage.json.tmp time: 21

Thanks.

[errm]

Thanks for the contribution...

[errm]

Thanks so much @ksauzz this has been merged!

[ksauzz]

Thanks!!