arcivanov
commented
6 years ago Async runs in a separate thread and blocks on a queue waiting for records to appear, starting to send it immediately as lock unblocks. How soon, depends on thread switching of the particular version of python implementation, but async means there are virtually no guarantees (although usually it's milliseconds).
Furthermore, in the current version of fluent-logger-python there is no guarantee at all, as guaranteed delivery of Protocol v1 is not used.
How/when does the async handler decide when to flush its buffers? What's the maximum time it would take for any buffered data to pumped to its outbound TCP socket?
We send our logs to a centralized log collector, that then ships (via HTTP) logs to Sumologic for indexing and have seen lag times of up to a few minutes for logs to appear in Sumo searches from the time a logger was invoked; I understand there's unavoidable end-to-end network delay, but would like to get a sense of the local guarantees.