Create a blueprint with a Grok task to show how to replace Logstash

kestra-io / docs

Documentation for Kestra — an event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.

https://kestra.io/docs

68 stars 60 forks source link

Create a blueprint with a Grok task to show how to replace Logstash #1600

Open loicmathieu opened 1 month ago

loicmathieu commented 1 month ago

Issue description

With the new io.kestra.plugin.transform.grok.TransformValue task from the Plugin Transform, we can replace Logstash by Kestra!

A blueprint would be great to see how we can handle a log file, parse it with the Grok task and save the parsed log to Elasticsearch.

It can either be done by providing a log file or analyzing logs in realtime from a Kafka real-time trigger.

shrutimantri commented 1 day ago

Blocked on https://github.com/kestra-io/plugin-transform/issues/7 and https://github.com/kestra-io/plugin-transform/issues/8