argoproj / argo-workflows

Workflow Engine for Kubernetes
https://argo-workflows.readthedocs.io/
Apache License 2.0
14.91k stars 3.18k forks source link

Allow `dependencies` to be templated #9297

Open DjokerR opened 2 years ago

DjokerR commented 2 years ago

Checklist

* [ ] Double-checked my configuration. * [ ] Tested using the latest version. * [ ] Used the Emissary executor. ## Summary how to use loops dags dependency, its not to unmashl What version are you running? argo: v3.3.7 kubernetes: v1.21.6 ## Diagnostics Paste the smallest workflow that reproduces the bug. We must be able to run the workflow. ```yaml apiVersion: argoproj.io/v1alpha1 kind: Workflow metadata: generateName: loops- spec: entrypoint: loop-example templates: - name: loop-example steps: - - name: print-message template: whalesay arguments: parameters: - name: tag value: "{{item.tag}}" - name: image value: "{{item.image}}" - name: dependen value: "{{item.dependencies}}" # WithParam: "{{steps.cat.outputs.result}}" withItems: [ { image: 'debian', tag: '11111' ,dependencies: 'debian' },{ image: 'centos', tag: 'tag222',dependencies: 'debian' },{ image: 'alpine', tag: 'tag333',dependencies: 'centos' } ] # invoke whalesay once for each item in parallel - name: whalesay inputs: parameters: - name: tag - name: image - name: dependen dag: tasks: - name: "{{inputs.parameters.image}}" template: at-tempalte # dependencies: ["{{inputs.parameters.dependen}}"] dependencies: - "{{inputs.parameters.dependen}}" #two ways is invalid arguments: parameters: - name: tag value: "{{inputs.parameters.tag}}" - name: image value: "{{inputs.parameters.image}}" - name: dependen value: "{{inputs.parameters.dependen}}" - name: at-tempalte inputs: parameters: - name: tag - name: image - name: dependen container: image: alpine:3.7 command: [echo, "{{inputs.parameters.dependen}}"] ``` ```bash Failed to submit workflow: templates.loop-example.steps[0].print-message templates.whalesay sorting failed: invalid dependency placeholder-9 ``` ---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

juliev0 commented 2 years ago

So, I started looking at this and there may possibly be a bug here. It looks like the issue occurs when validating the Workflow, before it gets run in the TopologicalSorting() function. I think we can prove it's a bug if I can comment out the Validate functionality and get it to successfully run after that.

However, do you have a good solution for the issue that the debian task seems to be dependent on itself? (unless I'm reading it wrong) Can the Workflow spec be modified?

sarabala1979 commented 2 years ago

@DjokerR the above example renders task name in runtime and validator doesn't have any visibility on runtime values. you can add functionality in the validator to create a placeholder for {{}} template names and dependency. @juliev0 This issue looks like an enhancement for me. Argo never supports this usecase .

juliev0 commented 2 years ago

Okay, thanks for looking. I have changed it to "enhancement" then.