argoproj / argo-workflows

Workflow Engine for Kubernetes
https://argo-workflows.readthedocs.io/
Apache License 2.0
14.85k stars 3.17k forks source link

Error when trying to pass Env Vars as a parameter to a template, cannot unmarshal object into Go struct #8827

Open RoryDoherty opened 2 years ago

RoryDoherty commented 2 years ago

Checklist

* [x] Double-checked my configuration. * [x] Tested using the latest version. * [x] Used the Emissary executor. ## Summary What happened/what you expected to happen? Tried to pass a parameter as a json/list of env vars but get the following error when submitting the workflow: ``` Unsuccessful HTTP response: json: cannot unmarshal object into Go struct field ScriptTemplate.workflow.spec.templates.script.env of type []v1.EnvVar ``` I have tried various assortments of quotes, yaml continuation using | and > According to this thread on slack it did work at some stage so this may be a regression: https://cloud-native.slack.com/archives/C01QW9QSSSK/p1628785975367900 What version are you running? v3.3.5 ## Diagnostics Paste the smallest workflow that reproduces the bug. We must be able to run the workflow. ```yaml apiVersion: argoproj.io/v1alpha1 kind: Workflow metadata: generateName: demo- namespace: argoci spec: entrypoint: demo templates: - name: demo dag: tasks: - name: env-var-test template: run-test arguments: parameters: - name: vars value: | [ {name: "Foo", value: "bar"}, {name: "Bar", value: "foo"}, {name: "zoo", value: "animal"} ] - name: run-test inputs: parameters: - name: vars script: image: ubuntu:18.04 imagePullPolicy: Always command: [bash] workingDir: "/src" env: {{inputs.parameters.vars}} source: | env ``` # Logs from the workflow controller: There were no logs in the workflow controller when I submitted the workflow via the UI Is there a trace logs setting I can enable? ---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

sandeepk8s commented 2 years ago

@RoryDoherty Try this

script:
   image: ubuntu:18.04
   command:
     - bash
   workingDir: /src
   env:
     - name: a
       value: '{{inputs.parameters.vars}}'
   source: |
     echo $a
RoryDoherty commented 2 years ago

@sandeepitachi Thanks, I tried that and yes that works, I get this output:

[ {name: "Foo", value: "bar"}, {name: "Bar", value: "foo"}, {name: "zoo", value: "animal"} ]

However I'm not trying to pass that as a string, I want to be able to set the env vars for a template within the dag task. So in my original example I want "Foo" "Bar" and "zoo" to be environment variables I can access within the script section

RoryDoherty commented 2 years ago

I can work around this for now using the following:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: demo-
spec:
  entrypoint: demo
  templates:
    - name: demo
      dag:
        tasks:
          - name: env-var-test
            template: run-test
            arguments:
              parameters:
                - name: vars
                  value: |
                    export Foo="bar"
                    export Bar="Foo"

    - name: run-test
      inputs:
        parameters:
          - name: vars
      script:
        image: ubuntu:18.04
        imagePullPolicy: Always
        command: [bash]
        workingDir: "/src"
        env: 
          - name: vars
            value: '{{inputs.parameters.vars}}'
        source: |
          echo $vars > /tmp/vars
          source /tmp/vars
          env

But it would definitely be good if we could set environment variables from the task and not just in the template

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

RoryDoherty commented 2 years ago

This issue still exists

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

RoryDoherty commented 2 years ago

issue still exists

vitalii-cidersecurity commented 2 years ago

This is a big problem for us as well, the issue still exists.

ddemydenko commented 2 years ago

the same problem

sarabala1979 commented 2 years ago

@RoryDoherty This is by design this is not a bug. Script Template is using v1.container spec. if v1.container will accept the array of EnvVar. We can enhance the controller to parse the JSON and convert []envVar.

RoryDoherty commented 2 years ago

Thanks @sarabala1979 , I had seen a similar snippet in a slack thread as a working example in an earlier release which is why I opened it as a bug! I'm happy for it to be converted to an enhancement though, if you want to point me in the right direction of where the processing is I'd be happy to take a crack at it

tooptoop4 commented 2 years ago

any fix?

alexec commented 1 year ago

I don’t believe it is possible to change this behavior The templating merging works by marshaling the string into a struct. It cannot cater for arbitrary string due to whitespacing. Even if you managed to get it to work, any small change to white space would break it.

@RoryDoherty suggestion is looks like a good work-around.