argoproj / argo-workflows

Workflow Engine for Kubernetes
https://argo-workflows.readthedocs.io/
Apache License 2.0
15.08k stars 3.2k forks source link

Add structural templates as alternative to text placeholders #1684

Closed Ark-kun closed 2 years ago

Ark-kun commented 5 years ago

Fixes https://github.com/argoproj/argo/issues/1293 : "Argo should provide a (limited) set of functions available in templates to support basic tasks like string modification."

Argo's text placeholders are limited and is not flexible enough .

Even with the current very limited placeholder support, it's impossible to call some programs that use go format templates (e.g. docker) since their templates clash with Argo syntax.

I have experience with many custom text-based templating languages and all of them were painful or impossible to use for some cases.

The solution is to use structural templates. There are many advantages of structural templates over text-based templates:

With structural templates we can even support multiple templating engines.

Structural templates utilize the language that Argo workflows already use - YAML.

Here is a pretty complex example that uses concat and substring to construct the full image name out of the base image name and the shortened revision:

command: [
  docker,
  tag,
  {inputValue: source-image},
  {concat: [{inputValue: image-name}, ':', {substring: {str: {inputValue: revision}, count: 6}]}
]

P.S. {inputValue: source-image} is the structural equivalent of "{{inputs.parameters.source-image}}".

Here is an example of structural templating in a component/template definition: https://github.com/kubeflow/pipelines/blob/5043ec78afc63506ce70fca01d2936928cb0fedb/components/gcp/automl/create_dataset_for_tables/component.yaml#L109

alexec commented 2 years ago

This is fixed by Expression Tag Templates in v3.