Closed rantoniuk closed 2 years ago
rantoniuk
commented
2 years ago The problem seems to be in the LambdaInvokeAction, because after removing that in the generated .js file the problem disappears. However, LambdaInvokeAction() does not provide a way to specify region, nor does the parent Pipeline(). Moreover, this pipeline is not deployed in different regions, it's a single-region deployment that was previously working fine.
skinny85
commented
2 years ago Thanks for opening the issue @rantoniuk.
Can you show what is cfInvalidationLambda in your code?
rantoniuk
commented
2 years ago const cfInvalidationLambda = Function.fromFunctionArn(
this,
'cfInvalidationLambda',
'arn:aws:lambda:eu-west-1::function:CloudfrontInvalidationLambda',
);Yeah, so here's the issue.
I assume you skipped the account ID for anonymity reasons, but there is some account ID there, right? And is it the same account that the pipeline itself is in?
rantoniuk
commented
2 years ago No, I haven't skipped anything - this is exactly the ARN that is used, using the default ARN notation that assumes the current account id if not provided by default from CloudFormation].
I believe this is a regression because it all worked fine in earlier versions.
skinny85
commented
2 years ago Hmm, where did you get this concept of "default ARN"? The word "default" does not appear anywhere in the page you linked to. I've never seen ARNs used that way, and CDK doesn't interpret missing account the way you described.
I think if you switch from fromFunctionArn() to the new fromFunctionName() method, this should start working correctly again.
rantoniuk
commented
2 years ago Trying to switch to:
const cfInvalidationLambda = Function.fromFunctionName(
this,
'cfInvalidationLambda',
'CloudfrontInvalidationLambda',
);ends with this:
cdk/node_modules/@aws-cdk/aws-lambda/lib/function-base.ts:421
throw new Error('Cannot modify permission to lambda function. Function is either imported or $LATEST version.\n'
^
Error: Cannot modify permission to lambda function. Function is either imported or $LATEST version.
If the function is imported from the same account use `fromFunctionAttributes()` API with the `sameEnvironment` flag.
If the function is imported from a different account and already has the correct permissions use `fromFunctionAttributes()` API with the `skipPermissions` flag.Trying to use:
const cfInvalidationLambda = Function.fromFunctionAttributes(this, 'cfInvalidationLambda', {
functionArn: 'arn:aws:lambda:eu-west-1::function:CloudfrontInvalidationLambda',
sameEnvironment: true,
});ends with the same:
cdk/node_modules/@aws-cdk/aws-codepipeline/lib/pipeline.ts:1045
throw new Error('Pipeline stack which uses cross-environment actions must have an explicitly set region');Even if I hardcode the AWS account number with:
const cfInvalidationLambda = Function.fromFunctionArn(
this,
'cfInvalidationLambda',
`arn:aws:lambda:eu-west-1:${Accounts.DEVOPS}:function:CloudfrontInvalidationLambda`,
);which is coming from an enum, then it also does not work. And I still think this is a regression, because it was working before.
skinny85
commented
2 years ago Right. There's no mention of Lambda Function ARNs having an optional account.
Even if I hardcode the AWS account number with:
const cfInvalidationLambda = Function.fromFunctionArn( this, 'cfInvalidationLambda', `arn:aws:lambda:eu-west-1:${Accounts.DEVOPS}:function:CloudfrontInvalidationLambda`, );which is coming from an enum, then it also does not work. And I still think this is a regression, because it was working before.
"Does not work" - what is the error?
rantoniuk
commented
2 years ago There's no mention of Lambda Function ARNs having an optional account:
I would debate that, the above sentence says: "when you use account number" == when means it's optional.
Same error:
throw new Error('Pipeline stack which uses cross-environment actions must have an explicitly set region');OK. I think you have 2 ways of unblocking yourself:
Accounts.DEVOPS) to the Stack that contains the Pipeline (the account property of the env property of the Stack).Aws.ACCOUNT_ID as the account in the ARN of the Function you're importing, and see if that helps.rantoniuk
commented
2 years ago So this workaround works:
`arn:aws:lambda:eu-west-1:${Accounts.DEVOPS}:function:CloudfrontInvalidationLambda`,and also on the stack creation side:
new DynamicPipelineStack(app, 'dynamic-pipeline-stack', {
env: {
region: process.env.CDK_DEFAULT_REGION,
},
});Note that I'm passing only the region and not the account (and both should be picked up from default env, so still a bug).
must have an explicitly set regionCannot modify permission to lambda function. Function is either imported or $LATEST versionskinny85
commented
2 years ago Glad you got yourself unblocked @rantoniuk!
I'm keeping this open to investigate why using Function.fromFunctionName() does not work in this case.
rantoniuk
commented
2 years ago @skinny85 I just stumbled on it again with a very simple pipeline:
export class PipelineConstruct extends Construct {
pipeline: Pipeline;
constructor(scope: Construct, id: string) {
super(scope, id);
this.pipeline = new Pipeline(this, "TestPipeline", {});
const build = new PipelineProject(this, "TestPipelineCodeBuild", {
projectName: "MyTestPipelineCodeBuild",
});
const artifact = new Artifact();
const action = new CodeBuildAction({
actionName: "action",
project: build,
input: artifact,
});
this.pipeline.addStage({
stageName: "Build",
actions: [action],
});
this.pipeline.addStage({
stageName: "Deployment-DEV",
actions: [
new CloudFormationCreateUpdateStackAction({
runOrder: 1,
actionName: "DeployCF",
stackName: "TestStack",
adminPermissions: false,
// account: Accounts.TEST,
templatePath: artifact.atPath("TestStack.template.json"),
cfnCapabilities: [CfnCapabilities.NAMED_IAM, CfnCapabilities.AUTO_EXPAND],
}),
],
});
}
}
// bin/app.ts:
new DynamicPipelineStack(app, "PipelineStack");
With account: Accounts.TEST commented out, cdk synth works fine.
If I uncomment it, I get:
Error: Pipeline stack which uses cross-environment actions must have an explicitly set account
at Pipeline.getOtherStackIfActionIsCrossAccount (cdk/node_modules/aws-cdk-lib/aws-codepipeline/lib/pipeline.js:1:13697)
at Pipeline.getRoleFromActionPropsOrGenerateIfCrossAccount (cdk/node_modules/aws-cdk-lib/aws-codepipeline/lib/pipeline.js:1:12839)
at Pipeline.getRoleForAction (cdk/node_modules/aws-cdk-lib/aws-codepipeline/lib/pipeline.js:1:11476)
at Pipeline._attachActionToPipeline (cdk/node_modules/aws-cdk-lib/aws-codepipeline/lib/pipeline.js:1:7771)
at Stage.attachActionToPipeline (cdk/node_modules/aws-cdk-lib/aws-codepipeline/lib/private/stage.js:1:3087)
at Stage.addAction (cdk/node_modules/aws-cdk-lib/aws-codepipeline/lib/private/stage.js:1:1716)
at new Stage (cdk/node_modules/aws-cdk-lib/aws-codepipeline/lib/private/stage.js:1:678)
at Pipeline.addStage (cdk/node_modules/aws-cdk-lib/aws-codepipeline/lib/pipeline.js:1:6662)
at new PipelineConstruct (cdk/stacks/pipeline-stack.ts:185:19)
at new DynamicPipelineStack (cdk/stacks/pipeline-stack.ts:32:5)It gets funnier, when I specify both:
account: Accounts.TEST,
region: "eu-west-1",the error complains about region missing :-)
Error: Pipeline stack which uses cross-environment actions must have an explicitly set regioncdk version 2.38.1 (build a5ced21)
skinny85
commented
2 years ago Right. So, you're saying, in the CloudFormationCreateUpdateStackAction, that you want to deploy to the account Accounts.TEST. But, you're not specifying which account you want the pipeline itself to be in. Because of that, the Pipeline construct can't tell if this will be a "regular" CodePipeline, or a cross-account one, and hence why it raises this error.
The same thing goes for region.
rantoniuk
commented
2 years ago Hi Adam,
My point is that is should probably assume to use the current-region and current-account, shouldn't it?
If I don't set region and account and provide a cross-account role instead - it works fine. Moreover, I have a project working where:
role (instead of autogenerating one) from a DEVOPS accountCloudFormationCreateUpdateStackAction I am also setting the cross-account target TEST roleThis also works fine - but if I try to switch from setting roles explicitly to using account and roles, then I get the above error.
Following your response, I tried to set the env explicitly, but that did not help:
new DynamicPipelineStack(app, "PipelineStack", {
env: {
account: process.env.CDK_DEFAULT_ACCOUNT,
region: process.env.CDK_DEFAULT_REGION,
},
});Setting it to explicit values also doesn't fix it and I don't see any other place where it can be set (e.g. on the Pipeline() construct level), can you advise where it should be set?
skinny85
commented
2 years ago You can't leave account/region unset, or to "dynamic" values with CDK_DEFAULT_ACCOUNT/REGION, because this decision, whether a pipeline is cross-account/region, must be made at synth time, and synthesis doesn't know what AWS account you are running it for (and it shouldn't produce different results anyway depending on it - it should be deterministic, and always produce the same result from the same source).
So, it should be:
new DynamicPipelineStack(app, 'PipelineStack', {
env: {
account: Accounts.TEST,
region: 'eu-west-1',
},
});When you provide the role yourself, we actually ignore the region / account properties (it should be stated in their docs) - we basically trust you at that point that you wired everything correctly.
rantoniuk
commented
2 years ago I was missing passing the props in the super() statement, so that's why it didn't work when I tried it.
Thanks for detailed explanation @skinny85!
github-actions[bot]
commented
2 years ago Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.
BwL1289
commented
1 year ago @skinny85 I am also now experiencing this error trying to run a very simple pipeline, and can't get around it without explicitly removing the env. If I remove env, it bootstraps fine. When env is defined (explicitly, as you stated above), I get:
RuntimeError: Error: Pipeline stack which uses cross-environment actions must have an explicitly set region.
To be clear, I'm literally hardcoding the account # and region right now trying to debug this.
Here's my code:
pipeline = pipelines.CodePipeline(
self,
id=id,
cross_account_keys=True,
synth=synth,
self_mutation=False,
docker_enabled_for_synth=True,
)
application = ApplicationStage(
self,
env=Environment(
account="<omitted>",
region="<omitted>",
),
)I have also tried turning self_mutation=True and playing with various parameters with no luck.
I've also double checked @rantoniuk's issue to see if i've left out a **kwargs anywhere, and I don't believe I have.
Would appreciate any help.
skinny85
commented
1 year ago @BwL1289 you need to set the env in the Stack containing the pipeline - not in the Stage that the pipeline is deploying.
BwL1289
commented
1 year ago @skinny85 sincerely appreciate the response.
In the docs they show you can add an env to a stage:
import aws_cdk as cdk
from constructs import Construct
from aws_cdk.pipelines import CodePipeline, CodePipelineSource, ShellStep
from my_pipeline.my_pipeline_app_stage import MyPipelineAppStage
class MyPipelineStack(cdk.Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
pipeline = CodePipeline(self, "Pipeline",
pipeline_name="MyPipeline",
synth=ShellStep("Synth",
input=CodePipelineSource.git_hub("OWNER/REPO", "main"),
commands=["npm install -g aws-cdk",
"python -m pip install -r requirements.txt",
"cdk synth"]))
pipeline.add_stage(MyPipelineAppStage(self, "test",
env=cdk.Environment(account="111111111111", region="eu-west-1")))Stages also support envs:
:param env: Default AWS environment (account/region) for ``Stack``s in this ``Stage``. Stacks defined inside this ``Stage`` with either ``region`` or ``account`` missing from its env will use the corresponding field given here. If either ``region`` or ``account``is is not configured for ``Stack`` (either on the ``Stack`` itself or on the containing ``Stage``), the Stack will be *environment-agnostic*. Environment-agnostic stacks can be deployed to any environment, may not be able to take advantage of all features of the CDK. For example, they will not be able to use environmental context lookups, will not automatically translate Service Principals to the right format based on the environment's AWS partition, and other such enhancements. Default: - The environments should be configured on the ``Stack``s.Here's more of the boilerplate code:
class Pipeline(Stack):
def __init__(
self,
scope: Construct,
*,
id: str,
**kwargs,
):
super().__init__(scope, id, **kwargs)
source = codestar_connection_builder()
synth = synth_step(id, source)
pipeline = pipelines.CodePipeline(
self,
id=id,
cross_account_keys=True,
synth=synth,
self_mutation=False, # TODO: turn off for prod
docker_enabled_for_synth=True,
)
application = ApplicationStage(
self,
env=Environment(
account="<omitted>",
region="<omitted>",
),
)
app_stage = pipeline.add_stage(application)What am I missing?
skinny85
commented
1 year ago What am I missing?
Like I wrote above, the error is most likely because the Stack containing your CodePipeline construct needs the env parameter passed to it as well.
BwL1289
commented
1 year ago @skinny85 still no luck. Same code as above, but with the env set on the Application, and I removed the env on the ApplicationStage:
class Application(Stack):
@property
def function_name(self) -> CfnOutput:
return self._function_name
def __init__(self, scope: Construct, id: str = "Application", **kwargs) -> None:
super().__init__(scope, id, **kwargs)
func = LambdaTestConstruct(self)
self._function_name = CfnOutput(func, "FuncARN", value=func.function_name)
ApplicationStage(Stage):
@property
def umbrella_function_name(self):
return self._umbrella.function_name
def __init__(
self,
scope: Construct,
*,
env: Optional[Union[Environment, dict[str, Any]]] = None,
outdir: Optional[str] = None,
permissions_boundary: Optional[PermissionsBoundary] = None,
stage_name: Optional[str] = None,
id: str = "ApplicationStage",
) -> None:
super().__init__(
scope,
id=id,
env=env,
outdir=outdir,
permissions_boundary=permissions_boundary,
stage_name=stage_name,
)
self._umbrella = Application(
self,
env=Environment(
account="<omitted>",
region="<omitted>",
),
)I finally figured this out. The issue was that I had a stack that created my Codepipeline and I was passing that to a separate stack: that's what was causing the error. The attempt was to modularize pipelines so multiple could be deployed together.
In other words, I had another Stack that I was passing into a different Stack, and that "other" Stack gets passed to app.py.
Instead, if you want to use this approach, use a Construct, not a Stack, to create your CodePipeline, and pass that into a different Stack.
I hope this makes sense and helps someone down the road.
JulianM33
commented
1 year ago Using an aws_cdk.Environment instead of a dictionairy worked for me.
jacido
commented
4 months ago The issue was that I had a
stackthat created myCodepipelineand I was passing that to a separatestack: that's what was causing the error. The attempt was to modularize pipelines so multiple could be deployed together.In other words, I had another
Stackthat I was passing into a differentStack, and that "other"Stackgets passed toapp.py.Instead, if you want to use this approach, use a
Construct, not aStack, to create yourCodePipeline, and pass that into a differentStack.
Hi, would you happen to have a working example of this?
Describe the bug
cdk synththrows the below error after upgrading aws-cdk@1.x to the current version. The returned error message is non-specific, i.e.Offending code:
is this:
Expected Behavior
Previously working fine.
Current Behavior
Reproduction Steps
not working on: 1.152.0 (build 9487b39) working on: 1.127.0
Possible Solution
No response
Additional Information/Context
No response
CDK CLI Version
1.152.0 (build 9487b39)
Framework Version
No response
Node.js Version
v14.19.1
OS
MacOS
Language
Typescript
Language Version
"typescript": "~4.3.5"
Other information
No response