microsoft / azure-pipelines-yaml

Azure Pipelines YAML examples, templates, and community interaction
MIT License
1.19k stars 926 forks source link

matrix with dependson #553

Closed ozbillwang closed 2 years ago

ozbillwang commented 2 years ago

matrix is useful and reduce the codes a lot, especially run some silmiar jobs repeatly

strategy:
  matrix:
    job1:
      imageName: 'hello-1'
    job2:
      imageName: 'hello-2'
    job3:
      imageName: 'hello-3'

But these jobs run in parallel

My jobs are related each other, so I'd like to wait for job1 finished, then move on to job2, then job3.

How can I do that?

KonstantinTyukalov commented 2 years ago

Hi @ozbillwang, based on the issue description, I can suggest the following options:

The first option is to use the each keyword: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops#each-keyword

By using each keyword you can define the object parameter and iterate through it, each step in the cycle will be executed sequentially one after another. Here is a YAML snippet for example:

parameters:
- name: valuesOfJob
  type: object # you can see other types here: https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema#types
  default:
  - first
  - second
  - third
steps:
- ${{ each value in parameters.valuesOfJob }}:
  - script: echo ${{ value }}

Another option is to use the dependsOn parameter for each job, this solution requires configuring each job manually: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml#dependencies

Example:

jobs:
- job: job1
  steps:
  - script: echo 'first'
- job: job2
  dependsOn: job1
  steps:
  - script: echo 'second'

- job: job3
  dependsOn: job2
  steps:
  - script: echo 'third'

Also, if you still want to use matrix, you can set the maxParallel parameter value to 1, this will force the pipeline to run the jobs one by one, the job queue will be based on matrix definition (https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema#matrix)

Example:

strategy:
  matrix:
    first:
      value: 'first'
    second:
      value: 'second'
    third:
      value: 'third'
    maxParallel: 1
steps:
- script: echo $(value) # jobs queue: first, second, third

If you have additional questions about possible solutions, feel free to ask. Thanks!

ozbillwang commented 2 years ago

maxParallel: 1 is the setting I am looking for. Thanks.