fluent-plugins-nursery / fluent-plugin-concat

Fluentd Filter plugin to concatenate multiline log separated in multiple events.
MIT License
108 stars 33 forks source link

Allow ruby expressions in plugin fields #115

Open AndresPineros opened 2 years ago

AndresPineros commented 2 years ago

Problem

Allow adding the equivalent of "enable_ruby: true" for the concat plugin fields.

I want to create a dynamic multiline pipeline that lets people decide which pattern they want to use based on pod annotations. This means I need to do something like:

...
multiline_start_regexp: ${record["kubernetes"]["annotations"]["logging_pattern"]}
...

This would allow having self-service applications that can "subscribe" to different logging pipelines and pass configurations to it using the annotations. This means I don't need to create an almost identical pipeline for every application that realizes that it has multi-line exceptions.