search

grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
24k stars 3.46k forks source link

Add support for structured metadata to the fluentd output plugin #14298

Open dkulchinsky opened 1 month ago

dkulchinsky commented 1 month ago

Is your feature request related to a problem? Please describe.

Feature request

Describe the solution you'd like

We'd like to see support added to the fluentd output plugin for structured metadata, similarly to how it was done for fluent-bit out_loki

Describe alternatives you've considered

Switching to fluent-bit, however we rely on some functionality in fluentd that is not trivial and in some cases impossible to migrate to fluent-bit, so we think introducing feature parity to the two outputs would be great for users of both platforms (fluentd & fluent-bit)

Additional context

N/A

nabeelmohamed commented 1 month ago

hey @dkulchinsky can i work on this issue. would you be happy to guide me?

dkulchinsky commented 1 month ago

I'm not familiar enough with the plugin internals, but here's some relevant information.

Loki structured metadata explained here: https://grafana.com/docs/loki/latest/send-data/promtail/stages/structured_metadata/

Implementation of structured metadata support in fluentbit as reference: https://github.com/fluent/fluent-bit/pull/9048