Deployment of custom CDK resource containing AWS Step Function fails

pbv0 commented 2 years ago

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

v17.8.0

Amplify CLI Version

8.0.3

What operating system are you using?

Mac

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No manual changes made

Amplify Categories

custom

Amplify Commands

push

Describe the bug

When adding a Step Function resource to a CDK stack generated by amplify add custom (and updating the CDK dependencies of the stack to the latest 1.153.1), the following error comes up during amplify deploy:

CREATE_FAILED customcustomResource73fbfaba AWS::CloudFormation::Stack Wed Apr 27 2022 22:14:08 GMT+0200 (Central European Summer Time) Template error: Mapping named 'ServiceprincipalMap' is not present in the 'Mappings' section of template.

Expected behavior

Step Function resource should be deployed.

Reproduction steps

amplify init.
amplify add custom.
Update CDK dependencies of the custom resource to latest CDK v1 version (1.153.1) by changing package.json to:

{
  "name": "custom-resource",
  "version": "1.0.0",
  "description": "",
  "scripts": {
    "build": "tsc",
    "watch": "tsc -w",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "dependencies": {
    "@aws-amplify/cli-extensibility-helper": "^2.0.0",
    "@aws-cdk/core": "~1.153.1",
    "@aws-cdk/aws-stepfunctions": "~1.153.1"
  },
  "devDependencies": {
    "typescript": "^4.2.4"
  }
}

and running npm install.

Insert minimal AWS Step Function in the CDK stack by updating cdk-stack.ts to:

import * as cdk from "@aws-cdk/core";
import * as AmplifyHelpers from "@aws-amplify/cli-extensibility-helper";
import * as sfn from "@aws-cdk/aws-stepfunctions";

export class cdkStack extends cdk.Stack {
  constructor(
    scope: cdk.Construct,
    id: string,
    props?: cdk.StackProps,
    amplifyResourceProps?: AmplifyHelpers.AmplifyResourceProps
  ) {
    super(scope, id, props);
    /* Do not remove - Amplify CLI automatically injects the current deployment environment in this input parameter */
    new cdk.CfnParameter(this, "env", {
      type: "String",
      description: "Current Amplify CLI env name",
    });

    const startState = new sfn.Pass(this, "StartState");

    const exampleSfn = new sfn.StateMachine(this, "StateMachine", {
      definition: startState,
    });
  }
}

Run amplify deploy and confirm deployment to produce error.

GraphQL schema(s)

No response

Log output

No response

Additional information

The CDK stack above deploys without issue ouside of Amplify with CDK v1 and v2.

The CDK stack above also deploys without issue with CDK v1 versions 1.124.0 and 1.128.0 (but not the latest version 1.153.1) as an amplify custom resource.

The stack also deploys if we add an IAM role to the Step Function which should not be necessary according to the CDK docs (and is not necessary when using the other CDK versions mentioned above):

import * as cdk from "@aws-cdk/core";
import * as AmplifyHelpers from "@aws-amplify/cli-extensibility-helper";
import * as sfn from "@aws-cdk/aws-stepfunctions";
import * as iam from "@aws-cdk/aws-iam";

export class cdkStack extends cdk.Stack {
  constructor(
    scope: cdk.Construct,
    id: string,
    props?: cdk.StackProps,
    amplifyResourceProps?: AmplifyHelpers.AmplifyResourceProps
  ) {
    super(scope, id, props);
    /* Do not remove - Amplify CLI automatically injects the current deployment environment in this input parameter */
    new cdk.CfnParameter(this, "env", {
      type: "String",
      description: "Current Amplify CLI env name",
    });

    const serviceRole = new iam.Role(this, "Role", {
      assumedBy: new iam.ServicePrincipal("states.eu-west-1.amazonaws.com"),
    });

    const startState = new sfn.Pass(this, "StartState");

    const exampleSfn = new sfn.StateMachine(this, "StateMachine", {
      definition: startState,
      role: serviceRole,
    });
  }
}

ykethan commented 2 years ago

hey @pbv0, Thank you for reaching out. I observed that currently the Amplify managed policy[1] for the IAM profile does not provide the ServiceprincipalMap and service principal permission by default . Could you please let us know if ServiceprincipalMap and service principal have been added to your IAM role? REF: https://docs.aws.amazon.com/step-functions/latest/dg/tutorial-lambda-state-machine-cloudformation.html#lambda-state-machine-cfn-create-role if not could you please add the permissions and try deploying the application.

[1] https://docs.amplify.aws/cli/reference/iam/

johnf commented 2 years ago

I'm seeing the same issue and I suspect it is a cdk issue rather than an amplify one

MtthwBrwng commented 2 years ago

I'm also experiencing this issue, I've attempted to do some digging into how to provide ServiceprincipalMap & service principal to no avail. Is this something that needs to be configured in the IAM console? Are there any code examples showing how to fix this issue using the CDK SDKs using Amplify?

dreamorosi commented 2 years ago

I'm seeing the same issue and I suspect it is a cdk issue rather than an amplify one

Trying to deploy the same kind of component in CDK (1.153.1) alone (so without Amplify involved) doesn't show the same issue though:

Resources:
  StateMachineRoleB840431D:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Action: sts:AssumeRole
            Effect: Allow
            Principal:
              Service:
                Fn::Join:
                  - ""
                  - - states.
                    - Ref: AWS::Region
                    - .amazonaws.com
        Version: "2012-10-17"
    Metadata:
      aws:cdk:path: CdkSfnTestStack/StateMachine/Role/Resource
  StateMachine2E01A3A5:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      RoleArn:
        Fn::GetAtt:
          - StateMachineRoleB840431D
          - Arn
      DefinitionString: '{"StartAt":"PassState","States":{"PassState":{"Type":"Pass","End":true}}}'
    DependsOn:
      - StateMachineRoleB840431D
    Metadata:
      aws:cdk:path: CdkSfnTestStack/StateMachine/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Analytics: v2:deflate64:H4sIAAAAAAAA/02MTQrCMBCFz9J9OjWIiDuha0HqCcJ0xGntRDKJLkLublM3rt4f37NgD3uwzdl9tMVx7jL6QJBv0eFsei8aQ8Jo+rsMpD4FpOrXYeTIXoqpYNZIr3sSrJVCvjpVs15Eujh8sGzMfy6G3QJ58M9tqlpKMeJHgkm7tz3CCXbNpMxtSBJ5IRh++gW2retksgAAAA==
    Metadata:
      aws:cdk:path: CdkSfnTestStack/CDKMetadata/Default
    Condition: CDKMetadataAvailable
Conditions:
  CDKMetadataAvailable:
    Fn::Or:
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - af-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-northeast-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-northeast-2
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-southeast-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-southeast-2
          - Fn::Equals:
              - Ref: AWS::Region
              - ca-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - cn-north-1
          - Fn::Equals:
              - Ref: AWS::Region
              - cn-northwest-1
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-north-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-2
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-3
          - Fn::Equals:
              - Ref: AWS::Region
              - me-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - sa-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-east-2
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - us-west-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-west-2

ykethan commented 2 years ago

Hey, I was able to reproduce the error in my amplify application. marking this as bug.

Note: the issue occurs on "@aws-cdk/aws-stepfunctions": "~1.153.1" version.

johnf commented 2 years ago

OK, new theory.

Amplify is using the CDK API to call CDK it doesn't run cdk depoy as a shell command.

My yarn.lock in amplify/backend has @aws-cdk/* versions locked at 1.124.0 Whereas in amplify/custom/xxx I think we are all installing the latest versions of CDK.

So there is a version mismatch.

MtthwBrwng commented 2 years ago

It appears the error goes away when replacing new iam.ServicePrincipal("states.eu-west-1.amazonaws.com") with new iam.AccountRootPrincipal() on 1.154.0. I'm not too sure if that has any unattended effects but it works.

below is OPs code not throwing an error on 1.154.0

import * as cdk from "@aws-cdk/core";
import * as AmplifyHelpers from "@aws-amplify/cli-extensibility-helper";
import * as sfn from "@aws-cdk/aws-stepfunctions";
import * as iam from "@aws-cdk/aws-iam";

export class cdkStack extends cdk.Stack {
  constructor(
    scope: cdk.Construct,
    id: string,
    props?: cdk.StackProps,
    amplifyResourceProps?: AmplifyHelpers.AmplifyResourceProps
  ) {
    super(scope, id, props);
    /* Do not remove - Amplify CLI automatically injects the current deployment environment in this input parameter */
    new cdk.CfnParameter(this, "env", {
      type: "String",
      description: "Current Amplify CLI env name",
    });

    const serviceRole = new iam.Role(this, "Role", {
      assumedBy: new iam.AccountRootPrincipal(),
    });

    const startState = new sfn.Pass(this, "StartState");

    const exampleSfn = new sfn.StateMachine(this, "StateMachine", {
      definition: startState,
      role: serviceRole,
    });
  }
}

smcroskey commented 2 years ago

I recently ran into this issue as well, not using Amplify but using Terraform CDK. I think it may have to do with this block of code here? I was able to work around it by passing in a region option (CDK complains that it's deprecated and shouldn't be used) so that it would use this resolution instead 🤷‍♂️

joekiller commented 1 year ago

This commit seems to be the culprit: https://github.com/aws/aws-cdk/pull/17984. Last bugfix on the release notes: https://github.com/aws/aws-cdk/releases/tag/v1.137.0

Likely a combination of the dynamic mapping based on region depending on a region being included in the stack for synthesis. Perhaps the undefined region in the constructor blows it up. It might be work revisiting this revert

MtthwBrwng commented 1 year ago

Since this still hasn't been resolved and I recently found myself having this same issue, and rediscovering the whole process here is a snippet that works for aws-cdk v2; tested on amplify-cli v11.0.5.

Be sure to replace the region in the ServicePrincipal value, as well as the region option.

import * as cdk from 'aws-cdk-lib';
import * as AmplifyHelpers from '@aws-amplify/cli-extensibility-helper';
import {Construct} from 'constructs';
import * as sfn from "aws-cdk-lib/aws-stepfunctions"
import {StateMachineType} from "aws-cdk-lib/aws-stepfunctions"
import * as iam from 'aws-cdk-lib/aws-iam';

export class cdkStack extends cdk.Stack {
    constructor(scope: Construct, id: string, props?: cdk.StackProps, amplifyResourceProps?: AmplifyHelpers.AmplifyResourceProps) {
        super(scope, id, props);
        /* Do not remove - Amplify CLI automatically injects the current deployment environment in this input parameter */
        new cdk.CfnParameter(this, 'env', {
            type: 'String',
            description: 'Current Amplify CLI env name',
        });
        /* AWS CDK code goes here - learn more: https://docs.aws.amazon.com/cdk/latest/guide/home.html */
        const {projectName, envName} = AmplifyHelpers.getProjectInfo()

        const roleResourceNamePrefix = `YourServiceRole-${projectName}`;
        const role = new iam.Role(this, 'YourServiceRole', {
            assumedBy: new iam.ServicePrincipal('states.us-west-1.amazonaws.com', {
                region: "us-west-1"
            }),
            roleName: `${roleResourceNamePrefix}-${cdk.Fn.ref('env')}`
        });

        const definition = new sfn.Pass(this, "PassDef")

        const stateMachine = new sfn.StateMachine(this, 'YourServiceStateMachine', {
            definition,
            stateMachineName: `YourServiceStateMachine-${envName}`,
            timeout: cdk.Duration.hours(1),
            stateMachineType: StateMachineType.STANDARD,
            role: role,
        });

        role.addToPolicy(
            new iam.PolicyStatement({
                actions: ['*'],
                resources: [stateMachine.stateMachineArn],
            }),
        );
    }
}

aws-amplify / amplify-cli