search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.66k stars 3.92k forks source link

CodePipeline Artifact validation fails on valid use case #2549

Closed ghidi closed 5 years ago

ghidi commented 5 years ago

Describe the bug The validation mechanism for CodePipeline artifacts fails in certain cases that are valid.

Error: Validation failed with the following errors: Artifact 'buildArtifact1' was used as input before being used as output at Synthesizer.synthesize (/.../node_modules/@aws-cdk/cdk/lib/synthesis.js:22:23) at App.run (/.../node_modules/@aws-cdk/cdk/lib/app.js:44:31) at Object. (/.../bin/infra.js:8:5) at Module._compile (internal/modules/cjs/loader.js:759:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:770:10) at Module.load (internal/modules/cjs/loader.js:628:32) at Function.Module._load (internal/modules/cjs/loader.js:555:12) at Function.Module.runMain (internal/modules/cjs/loader.js:824:10) at internal/main/run_main_module.js:17:11

To Reproduce Create a pipeline that has the following stages and dependencies between actions: buildAction1 has run order 1, input sourceArtifact1 and output buildArtifact1 buildAction2 has run order 2, input sourceArtifact2, extra inputs buildArtifact1 and output buildArtifact2

stages: [ { name: 'Source', actions: [sourceAction1, sourceAction2] }, { name: 'Build', actions: [buildAction1, buildAction2] }, { .... } ]

Expected behavior The validation should pass. This is a valid pipeline that worked correctly before codepipeline.Artifact construct was introduced.

Version:

skinny85 commented 5 years ago

Hey @ghidi ,

thanks for reporting this. I confirm I was able to reproduce the issue locally, and it is indeed a bug. I'll be working on a fix today.

Thanks, Adam

skinny85 commented 5 years ago

PR with the fix submitted here: https://github.com/awslabs/aws-cdk/pull/2558

skinny85 commented 5 years ago

BTW, until we release the fix in the linked PR, you can work around the issue by moving buildAction2 to a new Stage after Build, something like:

new codepipeline.Pipeline(this, 'Pipeline', {
  stages: [
    {
      name: 'Source',
      actions: [sourceAction1, sourceAction2],
    },
    {
      name: 'Build',
      actions: [buildAction1],
    },
    {
      name: 'Build_temp',
      actions: [buildAction2],
    },
    {
      // ....
    }
  ],
  // ...
});
stefanfreitag commented 5 years ago

@skinny85 I was running into this issue and the workaround helped me out. But I still have this issue when trying to setup two code pipelines in one stack with v.0.31.0 of aws-cdk. If required I can provide more details on my setup.

skinny85 commented 5 years ago

@stefanfreitag please do.

stefanfreitag commented 5 years ago

@skinny85 I tried to reduce the noise in my code a bit and uploaded it here https://gist.github.com/stefanfreitag/ab2c8a905d1f42ad210ca396f7452f83

skinny85 commented 5 years ago

@stefanfreitag are you using the same Artifact in both of your Pipelines? (What I mean, re-using the same Artifact object in the 2 Pipelines?)

stefanfreitag commented 5 years ago

@skinny85 I updated the gist by attaching the buildspec I used in the last stage/ action of the first pipeline.

To answer you question: To my understanding yes. If you look at lines 42 and 43 of the gist there are the calls to the two methods for setting up the pipelines. I pass the same artifact "imageDefinitionArtifact" to both methods.

skinny85 commented 5 years ago

That's not really supported in CodePipeline. You can't share artifacts between Pipelines. So I believe the validation error you've received is correct, and not related to the bug fixed in this issue.

stefanfreitag commented 5 years ago

@skinny85 Thanks for looking to this! I will then try to handle this differently.

amzar96 commented 8 months ago

Hi @skinny85 ,

I am facing the same issue now when creating CodeBuildAction

Action 'CodeBuild-xxx' is using an unnamed input Artifact, which is not being produced in this pipeline
def codepipeline_action(self, codebuild_project: list, image_name):
    list_cb_actions = []
    source_output = aws_codepipeline.Artifact()

    for project in codebuild_project:
        build_action = aws_codepipeline_actions.CodeBuildAction(
            action_name=f"CodeBuild-{str(project[1]).title()}",
            project=project[0],
            execute_batch_build=False,
            combine_batch_build_artifacts=False,
            input=source_output,
            outputs=[aws_codepipeline.Artifact()],
            environment_variables={
                "IMAGE_REPO_NAME": aws_codebuild.BuildEnvironmentVariable(
                    value=image_name
                ),
                "AWS_ACCOUNT_ID": aws_codebuild.BuildEnvironmentVariable(
                    value=self.AWS_ACCOUNT
                ),
            },
        )

        list_cb_actions.append(
            aws_codepipeline.StageProps(
                stage_name=f"{str(project[1]).title()}", actions=[build_action]
            )
        )

    return list_cb_actions

what is the issue here? I can reuse source_output right?

skinny85 commented 8 months ago

Hi @amzar96,

I think the problem is you need to pass a name to the Artifact when creating it, so something like:

    source_output = aws_codepipeline.Artifact('Source')

Let me know if that fixes your problem!