search

fluent / fluent-operator

Operate Fluent Bit and Fluentd in the Kubernetes way - Previously known as FluentBit Operator
Apache License 2.0
555 stars 229 forks source link

Support elastic_data_stream #1190

Closed fschlager closed 3 weeks ago

fschlager commented 1 month ago

What this PR does / why we need it:

This PR adds support for Elasticsearch's data streams:

A data stream lets you store append-only time series data across multiple indices while giving you a single named resource for requests. Data streams are well-suited for logs, events, metrics, and other continuously generated data.

The underlying fluentd plugin already supports this, with this PR we can make use of it in the usual fluent-operator context.

This is designed to be a separate plugin to keep things simple, to reflect how this is treated by fluentd itself and to not break existing configurations.

Which issue(s) this PR fixes:

Fixes #1170

Does this PR introduced a user-facing change?

Add support for `elasticDataStream` plugin

Additional documentation, usage docs, etc.:

benjaminhuo commented 3 weeks ago

@fschlager Thanks, that's big enhancement!