search

scalyr / scalyr-fluentd

The fluentd plugin for inserting log messages and metrics in Scalyr.
Apache License 2.0
6 stars 5 forks source link

add ensure_log_order config #14

Open shivaken opened 4 years ago

shivaken commented 4 years ago

This PR adds an option 'ensure_log_order' to enable/disable a block of code to 'ensure timestamp is at least 1 nanosecond greater than the last one'.

This change might be needed in cases like us, sending log to Scalyr through a fluentd setup for routing purpose instead of sending it from server instances directly.

imron commented 4 years ago

Thanks for submitting this.

This code is there because until recently, the way Scalyr stored logs on the server required each log coming from a single Scalyr session (represented in the plugin by @session) to have an timestamp larger than any previous logs in that session (regardless of which log file/source it came from). Log lines failing to meet that condition were discarded.

This restriction was recently lifted, but we're still testing it out to confirm that everything works.

Once fully verified, this entire section of code will be removed, so for now, we'll likely hold off on merging it.