search

fluent-plugins-nursery / fluent-plugin-systemd

This is a fluentd input plugin. It reads logs from the systemd journal.
Apache License 2.0
153 stars 43 forks source link

Duplicate entries from journald in 0.0.11 #72

Closed loburm closed 5 years ago

loburm commented 5 years ago

Our team have found the issue in one of our fluentd images. Steps to reproduce:

  1. Create file /etc/fluent/fluent.conf with content 
    
<source>
type systemd
filters [{ "_SYSTEMD_UNIT": "docker.service" }]
pos_file /var/log/gcp-journald-docker.pos
read_from_head true
tag docker
</source>
@type stdout 


2. Run command: docker run -ti --rm -v /etc/fluent/fluent.conf:/etc/fluent/fluent.conf -v /usr/lib64:/host/lib -v /var/log:/var/log --network=host gcr.io/google-containers/fluentd-gcp:2.0.18

It's going to print latest message from docker every second.

If use version 2.0.17 instead we don't see the same problem. Biggest difference between those two images is systemd gem (0.0.11 vs 0.0.9)
errm commented 5 years ago

Do you have some sample of the fluentd logs you could share ... it sounds perhaps like something is crashing/erring and its just retried every second ...