kestra-io / kestra

Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
https://kestra.io
Apache License 2.0
7.15k stars 429 forks source link

Dynamic execution labels from triggers #3876

Open loicmathieu opened 1 month ago

loicmathieu commented 1 month ago

Feature description

Currently, you can pass execution labels when starting an execution from the UI/API or a Webhook. But you cannot do that from a Trigger as triggers automatically start executions so the list of labels is static and not dynamic.

But there is interesting use case for that, for example to add some identifier in a label extracted from the trigger message.

What we can do is to allow to define dynamic labels in a trigger that will allow using Pebble expression so the trigger output will be available to create the labels.

For example:

id: trigger
type: io.kestra.plugin.jdbc.postgres.ReatimeTrigger
[...]
labels:
  key1: static value
  key2: "{{json(trigger.data).productId}}"
anna-geller commented 1 month ago

Thinking more, couldn't you do this instead?

id: myflow
namespace: dev
labels:
  key1: static value
  key2: "{{json(trigger.data).productId}}"
tasks:
...
triggers:
...

There is also a Labels task, so it's already possible

loicmathieu commented 1 month ago

@anna-geller we cannot do as you propose as flow labels are static labels not dynamic, we copy flow labels into the execution at run time but flow labels are also used to filter flows for ex.