(pipelines): allow builds to happen after UpdatePipeline

[darylteo]

Describe the feature

Ability to specify an additional build step after synth and after self-update and use this as a source for later Stages.

Use Case

Pretty frustrating that I can't do this - I am creating a Stack, which I put into a Stage. This works, except that

• I have to cdk synth in my Pipeline
• Which means that I have to build my source before performing the synth in order for the stack to pull Assets from the artifacts created by those tasks
• But the self-update does not trigger when I push updates to the build because the build has to happen first before the self-update. This is counter-intuitive. I have to manually perform stack updates through cdk deploy from my machine instead. and then when I push to my repo, it triggers another build again.

The intuitive solution if you're new to CDK (like I am) is to add a Wave to the Pipeline, before adding the Stage.

But the resulting Pipeline adds the "Assets" stage in between the UpdatePipeline stage and the Wave and there's no way to specify the Steps in the Wave as Outputs for the stack deployment Stage.

Proposed Solution

Allow us to specify a CodeBuildStep as a source for later Stacks I guess? 🤷‍♂️

Other Information

I've tried the following based on other topics along the same lines

• first calling pipeline.buildPipeline() then accessing the underlying pipeline to add an additional CodeBuildAction after the "self-update" action
• then trying to update the FileAsset CodeBuildActions to use different input
  • this unfortunately fails as the CodeBuildActions are immutable after they are created

    const buildArtifact = new Artifact('custom_output')
    const codeBuildAction = new CodeBuildAction({
      actionName: 'Build',
      project,
      input: sourceArtifact,
      outputs: [
        buildArtifact,
      ],
      runOrder: 2,
    })

    // pipeline.pipeline.stages[2] is the self-update stage. setting runOrder to 2 makes this run after the self-update.
    pipeline.pipeline.stages[2].addAction(codeBuildAction)

    for (const action of pipeline.pipeline.stages[3].actions) {
      // neither of this seems to work
      (action as unknown as any).props.inputs = [
        buildArtifact,
      ];

      (action as unknown as any).providedActionProperties.inputs = [
        buildArtifact,
      ]
    }

Acknowledgements

• [ ] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

CDK version used

2.133

Environment details (OS name and version, etc.)

os x

[pahud]

Actually in the synth stage you can define your custom commands before the cdk synth like

    const pipeline = new pipelines.CodePipeline(this, 'Pipeline', {
      synth: new pipelines.ShellStep('Synth', {
        input,
        }),
        commands: [
          // do what ever you want here
          'npm run build',
          'npx cdk synth',
        ],
      }),
    });

   Is this something you expect?

[darylteo]

That is in fact what I'm doing now but from my understanding this has 2 drawbacks:

• if the build breaks, the pipeline is broken completely and cannot self update and requires manual intervention via cdk deploy (because the buildspec is baked into the pipeline). Having the finicky build parts after the synth would somewhat mitigate against this. That being said when I mull over this, I realise that there is a chicken and egg problem as the deployment files need to exist to be put into the cdk.out artifact
• the 2nd is that if the build pipeline does have changes, this triggers the long build twice - once to build the cdk.out which then self updates and terminates the current Execution. It then seems to spawn a new execution which requires a build again.

The other thing which is kind of annoying is the requirement of the Assets stage - I've already got all the artifacts I need in the same place as the output of the build... why does it need 7 CodeBuildActions to copy the files i already have into separate buckets? 😅 is there any way this could be avoided as it seems to increase build time significantly and also sporadically fails due to too many concurrent codebuild actions.

Perhaps I've been looking at this philosophically from the wrong angle.