Missing log messages during pod startup

[felixbuenemann]

The deis logger misses messages that appear during pod startup until the logger starts tailing the container log.

The logs can be retrieved using kubectl logs, so maybe when a new pod is started, fluentd could be instructed to parse the entire log.

There appears to be an option to cause fluentd to read the entire log:

http://docs.fluentd.org/articles/in_tail#readfromhead

In order to avoid re-parsing old log entries if fluentd is restarted on the node, it could use a pos_file on a host volume.

[felixbuenemann]

This problem is especially bad because if a container crashes during startup, those messages are usually missing from deis logs.

[felixbuenemann]

I looked into the configuration and there is already a pos_file written to /var/log/containers.log.pos on the host, so it appears the only think left to do is changing read_from_head falseto read_from_head true in the source definition for "/var/log/containers/*.log".

[felixbuenemann]

Looks like I got the wrong repo, I'll open a new issue in deis/fluentd.