tektoncd / pipeline

A cloud-native Pipeline resource.
https://tekton.dev
Apache License 2.0
8.44k stars 1.77k forks source link

Variable to get a list of results of a task #4097

Open hokadiri opened 3 years ago

hokadiri commented 3 years ago

Feature request

if a i have a list of results

kind: Task
metadata:
  name: some-task
spec:
  results:
    - name: A
    - name: B
    - name: C
    - name: D
    - name: E
    - name: F

I want a variable that tells me the list so i can loop through it in in a task shell

      script: |
        # loop through $(results.list)

A variable which lists the results defined in a task

Use case

I have a list of 10 results , i want to set them to some value. Right now i have to

echo "X" > "$(results.A.path)" 
echo "Y" > "$(results.B.path)" 
echo "Z" > "$(results.C.path)" 
...

Would be great if i can loop through the results list/

Supports DRY principle.

bobcatfish commented 3 years ago

Thanks for opening this @hokadiri !

Are you able to share any more details around the use case you need this for? e.g. a specific Task with specific results would really help us understand why this is needed.

(I'm also wondering if in this case you'd prefer to be able to have an array or dictionary type result - but I think even if we supported those, we've been hesitant to support variable replacement for going from an array to a string b/c it's hard to say how it should be formatted - e.g. separated by commas? spaces? something else?)

hokadiri commented 3 years ago

Hi @bobcatfish, I have a tekton task that runs a tool which takes a git commit and outputs the list of tests to run with some additional metadata

{
  "test-a": {
    "cmd": "some cmd",
     ... some other metadata
  },
  "test-c": {
    "cmd": "some cmd",
    ... some other metadata
  },
  "test-d": {
    "cmd": "some cmd"
    ... some other metadata
  },
  "test-e": {
    "cmd": "some cmd"
    ... some other metadata
  },
}

Ideally I want to take this json and trigger concurrent task runs (One test per task).

Since results can't be lists/dictonaries, workaround is i'm creating a result for each test, looping thru and setting it.

To set it, I wanted to loop through the result list

For each test in $(results.list)
  if test in json:
    set result.test to true
  else:
    set result.test to false 

Ideally if results can support multiple types (not just string), then I can do: run task to generate json output -> convert json to results dict -> pass results dict to tekton taskB iterparams -> task-loop concurrently -> fan-in -> done

bobcatfish commented 3 years ago

ooooo thanks for the additional info!! i've actually just created 2 proposals that might help you out:

(Eventually I think we should allow even more complex types, e.g. arrays of dictionaries, etc.)

Maybe even just having array results support would be enough to unblock you for now? (it sounds like task-loops might be relevant to your interests also, if you wanted to run the same Task "for each" test)

hokadiri commented 3 years ago

@bobcatfish , Thanks for creating the proposals. Yeah having array results support - in conjunction with task-loops - would be enough to unblock me.

In general being able to convert a json/yaml file to an equivalent object tekton understands and vice versa would be a very helpful feature.

tekton-robot commented 2 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale with a justification. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close with a justification. If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

tekton-robot commented 2 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten with a justification. Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with /close with a justification. If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

ghost commented 2 years ago

/lifecycle frozen

Adding frozen lifecycle given its relevance to the array results & indexing TEP.