This PR adds multi-worker support to our fluentd plugin. Testing under load has shown that logs from a single docker container are always routed to the same fluentd worker.
This PR also removes the monotonically increasing timestamp restriction that was previously required by the server, and uses the tag name as the "thread_id", so that we can completely avoid the need for synchronization.
Finally, it updates the defaults to take in to account the increased maximum buffer size on the server.
Here is an example config file using 4 workers. With this config, I was able to transfer up to 20MB/s from a single Azure server based in US-East.
This PR adds multi-worker support to our fluentd plugin. Testing under load has shown that logs from a single docker container are always routed to the same fluentd worker.
This PR also removes the monotonically increasing timestamp restriction that was previously required by the server, and uses the tag name as the "thread_id", so that we can completely avoid the need for synchronization.
Finally, it updates the defaults to take in to account the increased maximum buffer size on the server.
Here is an example config file using 4 workers. With this config, I was able to transfer up to 20MB/s from a single Azure server based in US-East.