search

driskell / log-courier

The Log Courier Suite is a set of lightweight tools created to ship and process log files speedily and securely, with low resource usage, to Elasticsearch or Logstash instances.
Other
419 stars 107 forks source link

Log-courier new deploy picking log files from previous install #365

Closed psabhar closed 5 years ago

psabhar commented 6 years ago

Hello We redeploy our platform daily from scratch to test end to end orchestration and automation. During redeploy of application we dont clear the logs from previous install. We see that new log-courier installation picks the logs from old installation and hence we see duplicate logs been shipped to elasticsearch and hence we see spike in Kibana. Is there any way we can tel new installation of log-courier to resume the log shipping from state where old install of log-courier was before the redeploy happens?

driskell commented 6 years ago

If the /var/lib/log-courier structure still exists it should continue where it left off and not re-submit any files. If that structure is lost and re-created, Log Courier should begin from the END of the files and not start from scratch (unless the from-beginning flag was set when it was launched - but it defaults to off IIRC)

I can only think that the log files are being copied during re-deploy and then possibly put back. This would cause Log Courier (if it was resuming from existing /var/lib/log-courier structure) to see them as new files that just appeared as their identifiers would be different. If Log Courier is stopped whilst the copying/restoring of the logs happens and the /var/lib/log-courier is deleted, and it is then started, it should start from END of those files so not duplicate (any "new" start with no existing state only sends new logs, never existing - whereas any resume from existing state picks up everything it missed.)