search

fluent / helm-charts

Helm Charts for Fluentd and Fluent Bit
Apache License 2.0
374 stars 443 forks source link

Support service topologyKeys for fluent-bit chart #79

Open dmazhar-cogniance opened 3 years ago

dmazhar-cogniance commented 3 years ago

Hi team. Thanks for the chart! Do you consider supporting the topologyKeys for the fluent-bit service: https://github.com/fluent/helm-charts/blob/master/charts/fluent-bit/templates/service.yaml We're using fluent-bit running as a daemonset with extra port and tcp input to ingest structured logs from apps. topologyKey will allow to prefer local instance of fluent-bit to reduce inter-node traffic: https://kubernetes.io/docs/concepts/services-networking/service-topology/#prefer-node-local-zonal-then-regional-endpoints

naseemkullah commented 3 years ago

Hi @dmazhar-cogniance , sounds like it would be a nice feature to add, feel free to open a PR to add this. OOC why you send straight to fluent-bit tcp input from k8s apps instead of using the tail input that reads from the node's log file?

dmazhar-cogniance commented 3 years ago

OOC why you send straight to fluent-bit tcp input from k8s apps instead of using the tail input that reads from the node's log file?

We have apps running in a cluster with possible different logs formats. So to make it simple we have logs from the log files ingested "as is" and json structured logs with defined set of fields ingested via the tcp socket. But in general I think this could be done with tail input and labels filters too.