search

redpanda-data / connect

Fancy stream processing made operationally mundane
https://docs.redpanda.com/redpanda-connect/about/
8.15k stars 840 forks source link

Add support for Grok pattern definitions in external file(s) #424

Closed itkovian closed 3 years ago

itkovian commented 4 years ago

We have a lot of Grok patterns and we'd like to deploy the yaml file from our cfgmgmt system. It makes little sense to include the Grok patterns literally -- they are normal deployed using an RPM, we have tests etc for them.

Would it be possible to provide a pattern_source giving a directory (preferably) or file of patterns that should be included next to the default patterns?

Jeffail commented 4 years ago

Hey @itkovian, that's a reasonable addition. I've classed this as a bughancement as the current behaviour is odd and user hostile.

itkovian commented 4 years ago

Thanks!

Jeffail commented 3 years ago

Done https://github.com/Jeffail/benthos/commit/0b5664d5bf224f60032821c97ad27828f9423d18, planning to release later this week.