search

fabric8io / fluent-plugin-kubernetes_metadata_filter

Enrich your fluentd events with Kubernetes metadata
Apache License 2.0
351 stars 166 forks source link

Convert plugin to use filter method instead of filter_stream #268

Closed JSchuenke closed 4 years ago

JSchuenke commented 4 years ago

Hello!

We love this plugin and were using it to pull metadata from k8s. While looking for performance improvements, we noticed it was the only plugin in our config to not take advantage of the filter optimization done by fluentd because it was implementing the filter_stream method rather than the filter method. After a quick refactor over to the other method, we saw a 63% increase in our throughput.

We also added a Dockerfile we used to test that we found helpful and thought others would as well.

JSchuenke commented 4 years ago

After some further testing, our numbers were very off. We'll reopen if it makes sense. 👋