Add a new way to deploy fluentbit

fluent / fluent-operator

Operate Fluent Bit and Fluentd in the Kubernetes way - Previously known as FluentBit Operator

Apache License 2.0

580 stars 247 forks source link

Add a new way to deploy fluentbit #304

Open wenchajun opened 2 years ago

wenchajun commented 2 years ago

Is your feature request related to a problem? Please describe.

Since fluentbit upgraded to 1.9, many plugins have been added, such as the Prometheus Scrape Metrics plugin, but it is not convenient to collect some information if configured in the form of daemonset. So I would like to add the statefulset method to deploy fluentbit.

Describe the solution you'd like

To do this we need to add a new CRD, our CRD based on fluentbit's statefulset can be called collector.

Describe alternatives you've considered

To do this we may also need to modify the fluentbit-controller.

Additional context

No response

jcdauchy-moodys commented 2 years ago

How do you grab all pods logs if FB is not deployed as Daemonset ? FB reads files on each nodes to collect pods logs. If you have FB on only some K8s nodes, you'll be missing plenty of pod logs.

benjaminhuo commented 2 years ago

How do you grab all pods logs if FB is not deployed as Daemonset ? FB reads files on each nodes to collect pods logs. If you have FB on only some K8s nodes, you'll be missing plenty of pod logs. @jcdauchy-moodys

Fluent Operator support deploying fluent bit as a daemonset already. What we're talking about in this issue is the requirement to deploy fluentbit as a statefulset to scrap Prometheus metrics which doesn't require to be deployed as a daemonset

benjaminhuo commented 1 year ago

Add FluentBit collector is helpful in scenarios of collecting data through the network including the following input plugins: