freedomofpress / ansible-role-elk

Installs a turnkey ELK stack for log aggregation and analysis, with optional Riemann support for alerting
18 stars 13 forks source link

Update timestamps based on log data #21

Closed conorsch closed 8 years ago

conorsch commented 8 years ago

Right now the @timestamp field stores the time at which the log event was ingested by Logstash. For production systems that's largely OK, but in the event of an outage, or when onboarding a new client with historical log data, the timestamps will be flat-out wrong.

Let's use the date filter to update the timestamps where appropriate. We'll need different date logic per logfile, depending on the built-in time format.

conorsch commented 8 years ago

Handled for existing filters, so closing. The handful of open PRs already have date filters added as well—thanks, @ageis!