sprig functions do not work in a pipeline

[boiledfroginthewell]

Pre-requisites

• [X] I have double-checked my configuration
• [X] I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
• [X] I have searched existing issues and could not find a match for this bug
• [ ] I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

Expressions are not evaluated when sprig functions are used after a pipe (|) operator.

Outputs of the attached workflow:

with pipe:
{{=sprig.trim(" abc ") | sprig.upper()}}

without pipe:
ABC

with expr build-in functions:
ABC

Version(s)

v3.4.17, v3.5.11, latest git main branch (0dfecd6e3a18c7bb884000e1e98d8305440d8d49)

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: test-workflow-
spec:
  ttlStrategy:
    secondsAfterCompletion: 60
  entrypoint: main
  templates:
    - name: main
      script:
        image: busybox:1.37.0
        command:
          - ash
        source: |-
          echo '
            with pipe:
            {{=sprig.trim(" abc ") | sprig.upper()}}

            without pipe:
            {{=sprig.upper(sprig.trim(" abc "))}}

            with expr build-in functions:
            {{=trim(" abc ") | upper()}}
          '

Logs from the workflow controller

level=info msg="Processing workflow" Phase= ResourceVersion=219769 namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:29.134Z" level=info msg="Task-result reconciliation" namespace=argo numObjs=0 workflow=test-workflow-d56zl
time="2024-10-14T04:24:29.134Z" level=info msg="Updated phase  -> Running" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:29.134Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:29.135Z" level=info msg="was unable to obtain node for , letting display name to be nodeName" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:29.135Z" level=info msg="Pod node test-workflow-d56zl initialized Pending" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:29.140Z" level=info msg="Created pod: test-workflow-d56zl (test-workflow-d56zl)" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:29.140Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:29.140Z" level=info msg=reconcileAgentPod namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:29.143Z" level=info msg="Workflow update successful" namespace=argo phase=Running resourceVersion=219772 workflow=test-workflow-d56zl
time="2024-10-14T04:24:30.133Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=219772 namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:30.133Z" level=info msg="Task-result reconciliation" namespace=argo numObjs=0 workflow=test-workflow-d56zl
time="2024-10-14T04:24:30.133Z" level=info msg="node changed" namespace=argo new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=test-workflow-d56zl old.message= old.phase=Pending old.progress=0/1 workflow=test-workflow-d56zl
time="2024-10-14T04:24:30.134Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:30.134Z" level=info msg=reconcileAgentPod namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:30.137Z" level=info msg="Workflow update successful" namespace=argo phase=Running resourceVersion=219783 workflow=test-workflow-d56zl
time="2024-10-14T04:24:31.188Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=219783 namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:31.188Z" level=info msg="Task-result reconciliation" namespace=argo numObjs=1 workflow=test-workflow-d56zl
time="2024-10-14T04:24:31.188Z" level=info msg="node changed" namespace=argo new.message= new.phase=Running new.progress=0/1 nodeID=test-workflow-d56zl old.message=PodInitializing old.phase=Pending old.progress=0/1 workflow=test-workflow-d56zl
time="2024-10-14T04:24:31.188Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:31.188Z" level=info msg=reconcileAgentPod namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:31.192Z" level=info msg="Workflow update successful" namespace=argo phase=Running resourceVersion=219790 workflow=test-workflow-d56zl
time="2024-10-14T04:24:33.117Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=219790 namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:33.118Z" level=info msg="Task-result reconciliation" namespace=argo numObjs=1 workflow=test-workflow-d56zl
time="2024-10-14T04:24:33.118Z" level=info msg="task-result changed" namespace=argo nodeID=test-workflow-d56zl workflow=test-workflow-d56zl
time="2024-10-14T04:24:33.118Z" level=info msg="node changed" namespace=argo new.message= new.phase=Running new.progress=0/1 nodeID=test-workflow-d56zl old.message= old.phase=Running old.progress=0/1 workflow=test-workflow-d56zl
time="2024-10-14T04:24:33.119Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:33.119Z" level=info msg=reconcileAgentPod namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:33.122Z" level=info msg="Workflow update successful" namespace=argo phase=Running resourceVersion=219797 workflow=test-workflow-d56zl
time="2024-10-14T04:24:33.124Z" level=info msg="cleaning up pod" action=terminateContainers key=argo/test-workflow-d56zl/terminateContainers
time="2024-10-14T04:24:34.121Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=219797 namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:34.121Z" level=info msg="Task-result reconciliation" namespace=argo numObjs=1 workflow=test-workflow-d56zl
time="2024-10-14T04:24:34.122Z" level=info msg="node unchanged" namespace=argo nodeID=test-workflow-d56zl workflow=test-workflow-d56zl
time="2024-10-14T04:24:34.122Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:34.122Z" level=info msg=reconcileAgentPod namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:34.127Z" level=info msg="cleaning up pod" action=terminateContainers key=argo/test-workflow-d56zl/terminateContainers
time="2024-10-14T04:24:35.210Z" level=info msg="Processing workflow" Phase=Running ResourceVersion=219797 namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:35.211Z" level=info msg="Task-result reconciliation" namespace=argo numObjs=1 workflow=test-workflow-d56zl
time="2024-10-14T04:24:35.211Z" level=info msg="node changed" namespace=argo new.message= new.phase=Succeeded new.progress=0/1 nodeID=test-workflow-d56zl old.message= old.phase=Running old.progress=0/1 workflow=test-workflow-d56zl
time="2024-10-14T04:24:35.211Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:35.211Z" level=info msg=reconcileAgentPod namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:35.211Z" level=info msg="Updated phase Running -> Succeeded" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:35.211Z" level=info msg="Marking workflow completed" namespace=argo workflow=test-workflow-d56zl
time="2024-10-14T04:24:35.214Z" level=info msg="Workflow update successful" namespace=argo phase=Succeeded resourceVersion=219802 workflow=test-workflow-d56zl
time="2024-10-14T04:24:35.215Z" level=info msg="Queueing Succeeded workflow argo/test-workflow-d56zl for delete in 1m0s due to TTL"
time="2024-10-14T04:24:35.223Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo/test-workflow-d56zl/labelPodCompleted
time="2024-10-14T04:24:36.125Z" level=info msg="cleaning up pod" action=killContainers key=argo/test-workflow-d56zl/killContainers
time="2024-10-14T04:25:36.000Z" level=info msg="Deleting garbage collected workflow 'argo/test-workflow-d56zl'"

Logs from in your workflow's wait container

time="2024-10-14T04:24:32.188Z" level=info msg="No output parameters"
time="2024-10-14T04:24:32.188Z" level=info msg="No output artifacts"
time="2024-10-14T04:24:32.188Z" level=info msg="S3 Save path: /tmp/argo/outputs/logs/main.log, key: test-workflow-d56zl/test-workflow-d56zl/main.log"
time="2024-10-14T04:24:32.188Z" level=info msg="Creating minio client using static credentials" endpoint="minio:9000"
time="2024-10-14T04:24:32.188Z" level=info msg="Saving file to s3" bucket=my-bucket endpoint="minio:9000" key=test-workflow-d56zl/test-workflow-d56zl/main.log path=/tmp/argo/outputs/logs/main.log
time="2024-10-14T04:24:32.192Z" level=info msg="Save artifact" artifactName=main-logs duration=3.351109ms error="<nil>" key=test-workflow-d56zl/test-workflow-d56zl/main.log
time="2024-10-14T04:24:32.192Z" level=info msg="not deleting local artifact" localArtPath=/tmp/argo/outputs/logs/main.log
time="2024-10-14T04:24:32.192Z" level=info msg="Successfully saved file: /tmp/argo/outputs/logs/main.log"
time="2024-10-14T04:24:32.197Z" level=info msg="Alloc=8205 TotalAlloc=14202 Sys=28773 NumGC=4 Goroutines=10"
time="2024-10-14T04:24:32.199Z" level=info msg="Deadline monitor stopped"

[boiledfroginthewell]

I patched the workflow controller to always show error messages from expr and then ran the attached workflow again. It seems . is not supported in pipelines by expr:

failed to compile:  unexpected token Operator(".") (1:28)
| sprig.trim(" abc ") | sprig.upper()
| ...........................^

If a function is assigned to a variable to avoid . in a pipeline, the evaluation succeeds:

{{=
    let sprig_upper = sprig.upper;
    sprig.trim(" abc ") | sprig_upper()
}}  # == ABC

[boiledfroginthewell]

I also found that usage of sprig functions is confusing in Argo Workflows. The Argo Workflows document just says

https://argo-workflows.readthedocs.io/en/stable/variables/

You can also use Sprig functions:

but examples in the Sprig document basically do not work in Argo Workflows:

1: Parentheses are required to call functions

# sprig document example: http://masterminds.github.io/sprig/strings.html#trim
{{=sprig.trim "   hello    "}}

expr error message:

failed to compile:  unexpected token String("   hello    ") (1:12)
| sprig.trim "   hello    "
| ...........^

# Correct syntax in Argo Workflows
{{=sprig.trim("   hello    ")}} # == hello

2: The pipe operator (|) passes a value to different argument positions

# In the sprig document, a left-hand side value is passed to the last argument of the function by the pipe operator
# sprig document example: http://masterminds.github.io/sprig/crypto.html#decryptaes
{{=
    let decryptAES = sprig.decryptAES;
    "30tEfhuJSVRhpG97XCuWgz2okj7L8vQ1s6V9zVUPeDQ=" | decryptAES("secretkey")
}}

expr error message:

illegal base64 data at input byte 8 (3:54)
|     "30tEfhuJSVRhpG97XCuWgz2okj7L8vQ1s6V9zVUPeDQ=" | sprig_decryptAES("secretkey")
| .....................................................^

# In expr, a left-hand side value is passed to the first argument of the function by the pipe operator
# Correct syntax:
{{=
    let decryptAES = sprig.decryptAES;
    "secretkey" | decryptAES("30tEfhuJSVRhpG97XCuWgz2okj7L8vQ1s6V9zVUPeDQ=")
}}  # == plaintext

The main reason is that Sprig document uses the template engine in the golang standard library while Argo Workflows uses expr.

---

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: test-workflow-
spec:
  ttlStrategy:
    secondsAfterCompletion: 1
  entrypoint: main
  templates:
    - name: main
      script:
        image: busybox:1.37.0
        command: 
          - ash
        source: |-
          echo '
            If a function is assigned to a variable to avoid `.` in a pipeline, the evaluation succeeds:
            {{=
                let sprig_upper = sprig.upper;
                sprig.trim(" abc ") | sprig_upper()
            }}  # == ABC

            sprig document example: http://masterminds.github.io/sprig/strings.html#trim
            {{=sprig.trim "   hello    "}}  # fail

            Correct syntax in Argo Workflows
            {{=sprig.trim("   hello    ")}} # == hello

            In the sprig document, a left-hand side value is passed to the last argument of the function by the pipe operator
            sprig document example: http://masterminds.github.io/sprig/crypto.html#decryptaes
            {{=
                let decryptAES = sprig.decryptAES;
                "30tEfhuJSVRhpG97XCuWgz2okj7L8vQ1s6V9zVUPeDQ=" | decryptAES("secretkey")
            }}  # fail in Argo Workflows

            In expr, a left-hand side value is passed to the first argument of the function by the pipe operator
            Correct syntax:
            {{=
                let decryptAES = sprig.decryptAES;
                "secretkey" | decryptAES("30tEfhuJSVRhpG97XCuWgz2okj7L8vQ1s6V9zVUPeDQ=")
            }}  # == plaintext
          "
          '