aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.58k stars 3.88k forks source link

(AWS CDK): What is the expected behavior for findInMap #14254

Closed jahtoe closed 3 years ago

jahtoe commented 3 years ago

:question: General Issue

How do I access mapping values

Given a template with a Mapping

Mappings:
  CidrMappings:
    vpc:
      CIDR: 10.0.0.0/16

The following code

  public String getCfnVPCMapping(CfnVPC cfnVPC) throws Exception {
    String resolvedCidrBlock = stack.resolve(cfnVPC.getCidrBlock()).toString();
    String mapping = "";
    LOG.debug("getCfnVPCMapping resolvedCidrBlock " + resolvedCidrBlock);
    if (resolvedCidrBlock.contains("Fn::FindInMap")) {
      Pattern pattern = Pattern.compile("\\{Fn::FindInMap=\\[([^,]+?), ([^,]+?), ([^]]+?)\\]", Pattern.CASE_INSENSITIVE);
      Matcher matcher = pattern.matcher(resolvedCidrBlock);
      if (matcher.find()) {
        String MapName = matcher.group(1);
        String TopLevelKey = matcher.group(2);
        String SecondLevelKey = matcher.group(3);
        LOG.debug("MapName " + MapName + " TopLevelKey " + TopLevelKey + " SecondLevelKey " + SecondLevelKey);
        if (stack.template.getMapping(MapName) != null) {
          mapping = stack.template.getMapping(MapName).findInMap(TopLevelKey, SecondLevelKey);
          LOG.debug("mapping " + mapping);
          String resolvedMapping = stack.resolve(mapping).toString();
          LOG.debug("resolvedMapping " + resolvedMapping);
        } else {
          LOG.warn("Unable to locate mapping " + MapName);
        }
      }
    }
    return mapping;
  }

Output the following logging

2021-04-19 15:35:44,063 DEBUG com.iriusrisk.iac.awscdk.Action [main] getCfnVPCMapping resolvedCidrBlock {Fn::FindInMap=[CidrMappings, vpc, CIDR]}
2021-04-19 15:35:44,064 DEBUG com.iriusrisk.iac.awscdk.Action [main] MapName CidrMappings TopLevelKey vpc SecondLevelKey CIDR
2021-04-19 15:35:44,065 DEBUG com.iriusrisk.iac.awscdk.Action [main] mapping ${Token[TOKEN.298]}
2021-04-19 15:35:44,066 DEBUG com.iriusrisk.iac.awscdk.Action [main] resolvedMapping {Fn::FindInMap=[CidrMappings, vpc, CIDR]}

I'm interested to get the value of the mapping: 10.0.0.0/16

But instead the findInMap function returns {Fn::FindInMap=[CidrMappings, vpc, CIDR]}

I'm wondering if that is the expected behavior?

Environment

Other information

Full template


---
AWSTemplateFormatVersion: 2010-09-09
Description: Example of EC2 Auto Scaling group using EC2 Spot Instances, automatically scaling
  based on SQS queue depth
Mappings:
  CidrMappings:
    vpc:
      CIDR: 10.0.0.0/16
  amazonLinuxAmi:
    ap-northeast-1:
      AMI: ami-0c3fd0f5d33134a76
    ap-northeast-2:
      AMI: ami-095ca789e0549777d
    ap-northeast-3:
      AMI: ami-0ee933a7f81beb045
    ap-south-1:
      AMI: ami-0d2692b6acea72ee6
    ap-southeast-1:
      AMI: ami-01f7527546b557442
    ap-southeast-2:
      AMI: ami-0dc96254d5535925f
    ca-central-1:
      AMI: ami-0d4ae09ec9361d8ac
    eu-central-1:
      AMI: ami-0cc293023f983ed53
    eu-north-1:
      AMI: ami-3f36be41
    eu-west-1:
      AMI: ami-0bbc25e23a7640b9b
    eu-west-2:
      AMI: ami-0d8e27447ec2c8410
    eu-west-3:
      AMI: ami-0adcddd3324248c4c
    sa-east-1:
      AMI: ami-058943e7d9b9cabfb
    us-east-1:
      AMI: ami-0b898040803850657
    us-east-2:
      AMI: ami-0d8f6eb4f641ef691
    us-west-1:
      AMI: ami-056ee704806822732
    us-west-2:
      AMI: ami-082b5a644766e0e6f
Metadata:
  Author:
    Description: Ran Sheinberg <ranshein@amazon.com>, Chad Schmutzer <schmutze@amazon.com>
  License:
    Description: 'Copyright 2019 Amazon.com, Inc. and its affiliates. All Rights Reserved.

      Licensed under the Amazon Software License (the "License"). You may not use this file
      except in compliance with the License. A copy of the License is located at

      http://aws.amazon.com/asl/

      or in the "license" file accompanying this file. This file is distributed on an "AS IS"
      BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
      License for the specific language governing permissions and limitations under the License.'
Outputs:
  autoScalingGroup:
    Description: The autoScalingGroup
    Value:
      Ref: autoScalingGroup
  awsRegionId:
    Description: The AWS Region ID your template was launched in
    Value:
      Ref: AWS::Region
  instanceProfile:
    Description: Instance profile ARN
    Value:
      Fn::GetAtt:
      - instanceProfile
      - Arn
  publicSubnet1:
    Description: Public subnet 1
    Value:
      Ref: publicSubnet1
  publicSubnet2:
    Description: Public subnet 2
    Value:
      Ref: publicSubnet2
  s3BucketId:
    Description: The S3 bucket ID
    Value:
      Ref: s3Bucket
  sqsQueueArn:
    Description: The SQS queue ARN
    Value:
      Fn::GetAtt:
      - sqsQueue
      - Arn
  vpc:
    Description: The VPC
    Value:
      Ref: vpc
Parameters:
  desiredCapacity:
    Default: 2
    Description: Desired capacity
    Type: Number
  maxSize:
    Default: 12
    Description: Maximum capacity
    Type: Number
  minSize:
    Default: 2
    Description: Mininum capacity
    Type: Number
Resources:
  attachGateway:
    DependsOn:
    - vpc
    - internetGateway
    Properties:
      InternetGatewayId:
        Ref: internetGateway
      VpcId:
        Ref: vpc
    Type: AWS::EC2::VPCGatewayAttachment
  autoScalingGroup:
    DependsOn:
    - launchTemplate
    - publicSubnet1
    - publicSubnet2
    - autoScalingServiceLinkedRole
    Properties:
      DesiredCapacity:
        Ref: desiredCapacity
      HealthCheckType: EC2
      MaxSize:
        Ref: maxSize
      MinSize:
        Ref: minSize
      MixedInstancesPolicy:
        InstancesDistribution:
          OnDemandBaseCapacity: 0
          OnDemandPercentageAboveBaseCapacity: 0
          SpotAllocationStrategy: capacity-optimized
        LaunchTemplate:
          LaunchTemplateSpecification:
            LaunchTemplateId:
              Ref: launchTemplate
            Version: 1
          Overrides:
          - InstanceType: t3.large
          - InstanceType: c4.large
          - InstanceType: c5.large
          - InstanceType: m4.large
          - InstanceType: m5.large
          - InstanceType: r4.large
          - InstanceType: r5.large
      VPCZoneIdentifier:
      - Ref: publicSubnet1
      - Ref: publicSubnet2
    Type: AWS::AutoScaling::AutoScalingGroup
  autoScalingPolicy:
    DependsOn:
    - autoScalingGroup
    - sqsQueue
    Properties:
      AutoScalingGroupName:
        Ref: autoScalingGroup
      PolicyType: TargetTrackingScaling
      TargetTrackingConfiguration:
        CustomizedMetricSpecification:
          Dimensions:
          - Name: QueueName
            Value:
              Fn::GetAtt:
              - sqsQueue
              - QueueName
          MetricName: ApproximateNumberOfMessagesVisible
          Namespace: AWS/SQS
          Statistic: Sum
          Unit: Count
        TargetValue: 2
    Type: AWS::AutoScaling::ScalingPolicy
  autoScalingServiceLinkedRole:
    Properties:
      AWSServiceName: autoscaling.amazonaws.com
      Description: Default Service-Linked Role enables access to AWS Services and Resources
        used or managed by Auto Scaling
    Type: AWS::IAM::ServiceLinkedRole
  cloudWatchLogsGroup:
    Properties:
      RetentionInDays: 7
    Type: AWS::Logs::LogGroup
  ec2FleetServiceLinkedRole:
    Properties:
      AWSServiceName: ec2fleet.amazonaws.com
      Description: Default EC2 Fleet Service Linked Role
    Type: AWS::IAM::ServiceLinkedRole
  instanceProfile:
    DependsOn:
    - instanceRole
    Properties:
      Path: /
      Roles:
      - Ref: instanceRole
    Type: AWS::IAM::InstanceProfile
  instanceRole:
    DependsOn:
    - sqsQueue
    - s3Bucket
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action:
          - sts:AssumeRole
          Effect: Allow
          Principal:
            Service:
            - ec2.amazonaws.com
        Version: 2012-10-17
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AmazonEC2RoleforSSM
      Path: /
      Policies:
      - PolicyDocument:
          Statement:
          - Action: sqs:*
            Effect: Allow
            Resource:
              Fn::GetAtt:
              - sqsQueue
              - Arn
          Version: 2012-10-17
        PolicyName:
          Fn::Join:
          - '-'
          - - Ref: AWS::StackName
            - sqsQueuePolicy
      - PolicyDocument:
          Statement:
          - Action:
            - logs:CreateLogGroup
            - logs:CreateLogStream
            - logs:PutLogEvents
            - logs:DescribeLogStreams
            Effect: Allow
            Resource: arn:aws:logs:*:*:*
          Version: 2012-10-17
        PolicyName:
          Fn::Join:
          - '-'
          - - Ref: AWS::StackName
            - cloudWatchLogsPolicy
      - PolicyDocument:
          Statement:
          - Action: s3:ListBucket
            Effect: Allow
            Resource:
            - Fn::Join:
              - ''
              - - 'arn:aws:s3:::'
                - Ref: s3Bucket
          Version: '2012-10-17'
        PolicyName:
          Fn::Join:
          - '-'
          - - Ref: AWS::StackName
            - s3BucketListPolicy
      - PolicyDocument:
          Statement:
          - Action: ec2:DescribeTags
            Effect: Allow
            Resource: '*'
          Version: '2012-10-17'
        PolicyName:
          Fn::Join:
          - '-'
          - - Ref: AWS::StackName
            - ec2DescribeTagsPolicy
      - PolicyDocument:
          Statement:
          - Action:
            - s3:PutObject*
            - s3:GetObject
            - s3:DeleteObject
            Effect: Allow
            Resource:
            - Fn::Join:
              - ''
              - - 'arn:aws:s3:::'
                - Ref: s3Bucket
                - /*
          Version: '2012-10-17'
        PolicyName:
          Fn::Join:
          - '-'
          - - Ref: AWS::StackName
            - s3BucketReadWritePolicy
      - PolicyDocument:
          Statement:
          - Action:
            - autoscaling:SetInstanceProtection
            - autoscaling:SetInstanceHealth
            Effect: Allow
            Resource: arn:aws:autoscaling:*:*:autoScalingGroup:*:autoScalingGroupName/*
          Version: '2012-10-17'
        PolicyName:
          Fn::Join:
          - '-'
          - - Ref: AWS::StackName
            - autoScalingGroupPolicy
    Type: AWS::IAM::Role
  internetGateway:
    DependsOn:
    - vpc
    Type: AWS::EC2::InternetGateway
  launchTemplate:
    DependsOn:
    - instanceProfile
    - s3Bucket
    - sqsQueue
    - cloudWatchLogsGroup
    Properties:
      LaunchTemplateData:
        IamInstanceProfile:
          Arn:
            Fn::GetAtt:
            - instanceProfile
            - Arn
        ImageId:
          Fn::FindInMap:
          - amazonLinuxAmi
          - Ref: AWS::Region
          - AMI
        TagSpecifications:
        - ResourceType: instance
          Tags:
          - Key: Name
            Value:
              Ref: AWS::StackName
        UserData:
          Fn::Base64:
            Fn::Sub: '#!/bin/bash -xe

              yum -y install git

              cd /root && git clone https://github.com/awslabs/ec2-spot-labs.git

              REGION=${AWS::Region} S3BUCKET=${s3Bucket} SQSQUEUE=${sqsQueue} CLOUDWATCHLOGSGROUP=${cloudWatchLogsGroup}
              bash /root/ec2-spot-labs/sqs-ec2-spot-asg/user-data.sh

              '
    Type: AWS::EC2::LaunchTemplate
  publicRoute:
    DependsOn:
    - publicRouteTable
    - internetGateway
    - attachGateway
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId:
        Ref: internetGateway
      RouteTableId:
        Ref: publicRouteTable
    Type: AWS::EC2::Route
  publicRouteTable:
    DependsOn:
    - vpc
    - attachGateway
    Properties:
      Tags:
      - Key: Name
        Value: Public Route Table
      VpcId:
        Ref: vpc
    Type: AWS::EC2::RouteTable
  publicSubnet1:
    DependsOn:
    - attachGateway
    Properties:
      AvailabilityZone:
        Fn::Select:
        - 0
        - Fn::GetAZs:
            Ref: AWS::Region
      CidrBlock:
        Fn::Select:
        - 0
        - Fn::Cidr:
          - Fn::FindInMap:
            - CidrMappings
            - vpc
            - CIDR
          - 256
          - 8
      MapPublicIpOnLaunch: true
      Tags:
      - Key: Name
        Value:
          Ref: AWS::StackName
      VpcId:
        Ref: vpc
    Type: AWS::EC2::Subnet
  publicSubnet1RouteTableAssociation:
    DependsOn:
    - publicRouteTable
    - publicSubnet1
    - attachGateway
    Properties:
      RouteTableId:
        Ref: publicRouteTable
      SubnetId:
        Ref: publicSubnet1
    Type: AWS::EC2::SubnetRouteTableAssociation
  publicSubnet2:
    DependsOn:
    - attachGateway
    Properties:
      AvailabilityZone:
        Fn::Select:
        - 1
        - Fn::GetAZs:
            Ref: AWS::Region
      CidrBlock:
        Fn::Select:
        - 1
        - Fn::Cidr:
          - Fn::FindInMap:
            - CidrMappings
            - vpc
            - CIDR
          - 256
          - 8
      MapPublicIpOnLaunch: true
      Tags:
      - Key: Name
        Value:
          Ref: AWS::StackName
      VpcId:
        Ref: vpc
    Type: AWS::EC2::Subnet
  publicSubnet2RouteTableAssociation:
    DependsOn:
    - publicRouteTable
    - publicSubnet2
    - attachGateway
    Properties:
      RouteTableId:
        Ref: publicRouteTable
      SubnetId:
        Ref: publicSubnet2
    Type: AWS::EC2::SubnetRouteTableAssociation
  s3Bucket:
    DependsOn:
    - sqsQueue
    - sqsQueuePolicy
    Properties:
      NotificationConfiguration:
        QueueConfigurations:
        - Event: s3:ObjectCreated:*
          Queue:
            Fn::GetAtt:
            - sqsQueue
            - Arn
    Type: AWS::S3::Bucket
  sqsQueue:
    Properties:
      VisibilityTimeout: 900
    Type: AWS::SQS::Queue
  sqsQueuePolicy:
    DependsOn:
    - sqsQueue
    Properties:
      PolicyDocument:
        Statement:
        - Action:
          - sqs:SendMessage
          Effect: Allow
          Principal:
            Service: s3.amazonaws.com
          Resource: '*'
        Version: 2012-10-17
      Queues:
      - Ref: sqsQueue
    Type: AWS::SQS::QueuePolicy
  vpc:
    Properties:
      CidrBlock:
        Fn::FindInMap:
        - CidrMappings
        - vpc
        - CIDR
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
      - Key: Name
        Value:
          Ref: AWS::StackName
    Type: AWS::EC2::VPC
...
jahtoe commented 3 years ago

Greetings @skinny85, I was wondering if you could look at the following as well Regards jahtoe

skinny85 commented 3 years ago

Hey @jahtoe ,

thanks for opening the issue. Can you explain what are you trying to achieve here?

I'm interested to get the value of the mapping: 10.0.0.0/16

Then I would suggest using Fn.findInMap("CidrMappings", "vpc", "CIDR").

Thanks, Adam

jahtoe commented 3 years ago

Hi @skinny85 , I'm trying to dynamically retrieve the value 10.0.0.0/16 from the mappings by using the following code

stack.template.**getMapping**(MapName).**findInMap**(TopLevelKey, SecondLevelKey);

Ideally I would like to be able to do this dynamically for any mapping.

findInMap takes two not null string parameters

Regads Jahtoe

skinny85 commented 3 years ago

So, the way Mappings are represented in CloudFormation is the Fn::FindInMap function. You can get it from either the CfnMapping class instance, like this:

var cfnMapping = stack.template.getMapping("CidrMappings");
var cirdr = cfnMapping.findInMap("vpc", "CIDR");

or you can construct the function directly:

var cirdr = Fn.findInMap("CidrMappings", "vpc", "CIDR");

In any case, you can pass cirdr as the value of a property of a different construct, and it will get resolved to what you need.

Hope this helps!

Thanks, Adam

jahtoe commented 3 years ago

Hi @skinny85,

Okay thanks for that response. I think the issue might be that I'm trying to access the value 10.0.0.0/16

In testing

LOG.debug("Fn.findInMap " + stack.resolve(Fn.findInMap(MapName, TopLevelKey, SecondLevelKey)).toString());

Fn.findInMap returns a token which outputs the following

Fn.findInMap {Fn::FindInMap=[CidrMappings, vpc, CIDR]}

I don't think then there is a way to access the value 10.0.0.0/16

Regards Jahtoe

skinny85 commented 3 years ago

Yes. That token that you see, at deployment time, will resolve to the value 10.0.0.0/16. So using it will achieve what you want here.

github-actions[bot] commented 3 years ago

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.