kestra-io / kestra

Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
https://kestra.io
Apache License 2.0
7.66k stars 470 forks source link

Feature Request: Importing of Pebble Macros within Workflow YAMLs #3760

Open japerry911 opened 4 months ago

japerry911 commented 4 months ago

Feature description

I reuse Pebble expressions all over my Workflow YAMLs, and could benefit from making them macros in a single file, and importing them to Workflow YAMLs.

If this is possible, this is a nice-to-have, it is not blocking anything.

Related Docs:

Kestra Macro

Pebble Import statement that might be relevant

Thank you 🙏

anna-geller commented 4 months ago

Asked the team to validate whether storing macros on a namespace level would be feasible:

resource "kestra_namespace" "example" {
  namespace_id  = "io.kestra.mynamespace"
  description   = "Friendly description"
  variables     = <<EOT
k1: 1
k2:
    v1: 1
EOT
  task_defaults = <<EOT
- type: io.kestra.core.tasks.log.Log
  values:
    message: first {{flow.id}}
- type: io.kestra.core.tasks.debugs.Return
  values:
    format: first {{flow.id}}
EOT
  macros = <<EOT
{% macro input(type, name) %}
    <input type="{{ type }}" name="{{ name }}" />
{% endmacro %}
{% macro my_macro(column, table) %}
    SELECT {{ column }}
    FROM {{ table }}
{% endmacro %}
EOT
}

in OSS, this would need to be defined in the configuration the same way as for taskDefaults