awslabs / aws-servicebroker

AWS Service Broker
Apache License 2.0
468 stars 131 forks source link

Support Elasticache Redis #152

Open hatless opened 5 years ago

hatless commented 5 years ago

This seems like an odd omission, given the otherwise broad array of datastore services available through the SB,

kferrone commented 4 years ago

So I am no Cloudformation expert, but I did take a stab at it. Basically the elasticache cloudformation spec has an engine property which can be memcached or redis. When set to redis some of the parameters options will change, e.g. version numbers. One big difference is redis can only have on AvalabilityZone whereas memcached can have many. So my strategy was manipulate the current elasticache template to omit the ability to have many AvalabilityZones.

I used these docs for reference: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/AWS_ElastiCache.html

Below is what I have so far, I hope it helps someone get it all the way to the end goal. Warning: This is not tested yet. I will update this comment along my journey.

AWSTemplateFormatVersion: 2010-09-09
Description: AWS Service Broker - Amazon ElastiCache for redis (qs-1nt0fs92c)
Metadata:
  AWS::ServiceBroker::Specification:
    Version: 1.0
    Tags:
    - AWS
    - RDS
    - elasticache
    - memcache
    Name: elasticache-redis
    DisplayName: Amazon ElastiCache
    LongDescription: Amazon ElastiCache is a web service that makes it easy to set
      up, manage, and scale distributed in-memory cache environments in the cloud.
      It provides a high performance, resizeable, and cost-effective in-memory cache,
      while removing the complexity associated with deploying and managing a distributed
      cache environment.
    ImageUrl: https://s3.amazonaws.com/awsservicebroker/icons/Database_AmazonElasticCache_LARGE.png
    DocumentationUrl: https://aws.amazon.com/documentation/elasticache/
    ProviderDisplayName: Amazon Web Services
    ServicePlans:
      production:
        DisplayName: Production
        Description: Configuration designed for production deployments
        LongDescription: Creates an Amazon ElastiCache for redis, optimised for
          production use
        Cost: https://aws.amazon.com/elasticache/pricing/
        ParameterValues:
          ClusterType: multi-node
          AllowVersionUpgrade: 'False'
          PortNumber: '6379'
          CidrBlocks: Auto
          CidrSize: '26'
      custom:
        DisplayName: Custom
        Description: Custom Configuration for Advanced deployments
        LongDescription: Creates an Amazon ElastiCache for redis with custom configuration
        Cost: https://aws.amazon.com/elasticache/pricing/
        ParameterValues: {}
  AWS::CloudFormation::Interface:
    ParameterGroups:
    - Label:
        default: Elasticache Settings
      Parameters:
      - ClusterType
      - NumCacheNodes
      - EngineVersion
      - CacheNodeType
      - AllowVersionUpgrade
    - Label:
        default: Network and Security
      Parameters:
      - VpcId
      - PortNumber
      - CidrBlocks
      - CidrSize
      - AccessCidr
    - Label:
        default: Maintenance
      Parameters:
      - PreferredMaintenanceWindowStartTime
      - PreferredMaintenanceWindowEndTime
    ParameterLabels:
      ClusterType:
        default: Cluster Type
      NumCacheNodes:
        default: Number of Cache Nodes
      EngineVersion:
        default: Engine Version
      CacheNodeType:
        default: Cache Node Type
      AllowVersionUpgrade:
        default: Allow Version Upgrade
      VpcId:
        default: Vpc ID
      PortNumber:
        default: Port Number
      CidrBlocks:
        default: CIDR Blocks
      CidrSize:
        default: CIDR Size
      AccessCidr:
        default: Access CIDR
      PubliclyAccessible:
        default: Publicly Accessible
      PreferredMaintenanceWindowStartTime:
        default: Preferred MaintenanceWindow Start Time
      PreferredMaintenanceWindowEndTime:
        default: Preferred Maintenance Window End Time
Parameters:
  VpcId:
    Description: The ID of the VPC to launch the Memcache cluster into
    Type: AWS::EC2::VPC::Id
  PreferredMaintenanceWindowDay:
    Description: The day of the week which ElastiCache maintenance will be performed
    Type: String
    Default: Mon
    AllowedValues:
    - Mon
    - Tue
    - Wed
    - Thu
    - Fri
    - Sat
    - Sun
  PreferredMaintenanceWindowStartTime:
    Description: The weekly start time in UTC for the ElastiCache maintenance window,
      must be less than PreferredMaintenanceWindowEndTime and cannot overlap with
      PreferredBackupWindow
    Type: String
    Default: 04:00
    AllowedValues:
    - 00:00
    - 01:00
    - 02:00
    - 03:00
    - 04:00
    - 05:00
    - 06:00
    - 07:00
    - 08:00
    - 09:00
    - '10:00'
    - '11:00'
    - '12:00'
    - '13:00'
    - '14:00'
    - '15:00'
    - '16:00'
    - '17:00'
    - '18:00'
    - '19:00'
    - '20:00'
    - '21:00'
    - '22:00'
  PreferredMaintenanceWindowEndTime:
    Description: The weekly end time in UTC for the ElastiCache maintenance window,
      must be more than PreferredMaintenanceWindowEndTime and cannot overlap with
      PreferredBackupWindow
    Type: String
    Default: 06:00
    AllowedValues:
    - 00:00
    - 01:00
    - 02:00
    - 03:00
    - 04:00
    - 05:00
    - 06:00
    - 07:00
    - 08:00
    - 09:00
    - '10:00'
    - '11:00'
    - '12:00'
    - '13:00'
    - '14:00'
    - '15:00'
    - '16:00'
    - '17:00'
    - '18:00'
    - '19:00'
    - '20:00'
    - '21:00'
    - '22:00'
  CidrBlocks:
    Description: comma seperated list of CIDR blocks to place ElastiCache into, must
      be the same quantity as specified in NumberOfAvailabilityZones. If auto is specified
      unused cidr space in the vpc will be used
    Type: CommaDelimitedList
    Default: Auto
  CidrSize:
    Description: Size of Cidr block to allocate if CidrBlocks is set to Auto.
    Type: String
    Default: '26'
  AccessCidr:
    Description: CIDR block to allow to connect to database
    Type: String
  CacheNodeType:
    Description: The compute and memory capacity of nodes in a cache cluster.
    Type: String
    AllowedValues:
    - cache.t2.small
    - cache.t2.medium
    - cache.m3.medium
    - cache.m3.large
    - cache.m3.xlarge
    - cache.m3.2xlarge
    - cache.m4.medium
    - cache.m4.large
    - cache.m4.xlarge
    - cache.m4.2xlarge
    - cache.m4.4xlarge
    - cache.m4.10xlarge
    - cache.r4.large
    - cache.r4.xlarge
    - cache.r4.2xlarge
    - cache.r4.4xlarge
    - cache.r4.8xlarge
    Default: cache.m4.large
  EngineVersion:
    Description: Family to be used with cluster or parameter group
    Type: String
    AllowedValues:
    - 5.0.0
    - 4.0.10
    - 3.2.10
    - 3.2.6
    - 3.2.4
    Default: 5.0.0
  NumCacheNodes:
    Description: The number of cache nodes in the cluster.
    Type: String
    Default: '3'
  ClusterType:
    Description: The type of cluster. Specify single-node or multi-node (default).  Number
      of nodes must be greater than 1 for multi-node
    Type: String
    AllowedValues:
    - single-node
    - multi-node
    Default: multi-node
  AllowVersionUpgrade:
    Description: Indicates that minor engine upgrades will be applied automatically
      to the cache cluster during the maintenance window. The default value is true.
    Type: String
    Default: 'True'
    AllowedValues:
    - 'True'
    - 'False'
  PortNumber:
    Description: The port number for the Cluster to listen on
    Type: Number
    Default: 6379
    MinValue: 1150
    MaxValue: 65535
Conditions:
  AutoCidrs:
    !Equals
    - !Select
      - 0
      - !Ref CidrBlocks
    - Auto
Resources:
  ElastiCacheCluster:
    Type: AWS::ElastiCache::CacheCluster
    Properties:
      AutoMinorVersionUpgrade: !Ref AllowVersionUpgrade
      CacheNodeType: !Ref CacheNodeType
      CacheParameterGroupName: !Ref ElasticacheParameterGroup
      CacheSubnetGroupName: !Ref ClusterSubnetGroup
      Engine: redis
      EngineVersion: !Ref EngineVersion
      NumCacheNodes: !Ref NumCacheNodes
      Port: !Ref PortNumber
      PreferredMaintenanceWindow: !Sub ${PreferredMaintenanceWindowDay}:${PreferredMaintenanceWindowStartTime}-${PreferredMaintenanceWindowDay}:${PreferredMaintenanceWindowEndTime}
      VpcSecurityGroupIds:
      - !Ref ClusterSecurityGroup
  ElasticacheParameterGroup:
    Type: AWS::ElastiCache::ParameterGroup
    Properties:
      CacheParameterGroupFamily: redis5.0
      Description: AWS Service Broker Elasticache cluster
      Properties: {}
  ClusterSubnetGroup:
    Type: AWS::ElastiCache::SubnetGroup
    Properties:
      Description: AWS Service Broker Elasticache cluster
      SubnetIds:
      - !Ref ClusterSubnet1
  ClusterSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - Fn::GetAZs: !Ref 'AWS::Region'
      VpcId: !Ref VpcId
      CidrBlock:
        !Select
        - 0
        - !If
          - AutoCidrs
          - !GetAtt AWSSBInjectedGetCidrs.CidrBlocks
          - !Ref CidrBlocks
  ClusterSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: !Sub Allow Client connections
      VpcId: !Ref VpcId
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: !Ref PortNumber
        ToPort: !Ref PortNumber
        CidrIp: !Ref AccessCidr
      SecurityGroupEgress:
      - IpProtocol: '-1'
        FromPort: '-1'
        ToPort: '-1'
        CidrIp: 0.0.0.0/0
  AWSSBInjectedLambdaZipsBucket:
    Type: AWS::S3::Bucket
    Properties:
      Tags: []
  AWSSBInjectedCopyZips:
    Type: AWS::CloudFormation::CustomResource
    Properties:
      ServiceToken: !GetAtt AWSSBInjectedCopyZipsLambda.Arn
      DestBucket: !Ref AWSSBInjectedLambdaZipsBucket
      SourceBucket: awsservicebrokeralpha
      Prefix: functions/
      Objects:
      - get_cidrs/lambda_function.zip
      - get_azs/lambda_function.zip
  AWSSBInjectedCopyZipsRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
        - Effect: Allow
          Principal:
            Service: lambda.amazonaws.com
          Action: sts:AssumeRole
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Path: /
      Policies:
      - PolicyName: lambda-copier
        PolicyDocument:
          Version: 2012-10-17
          Statement:
          - Effect: Allow
            Action:
            - s3:GetObject
            Resource:
            - arn:aws:s3:::awsservicebrokeralpha/*
          - Effect: Allow
            Action:
            - s3:PutObject
            - s3:DeleteObject
            Resource:
            - !Sub arn:aws:s3:::${AWSSBInjectedLambdaZipsBucket}/*
  AWSSBInjectedCopyZipsLambda:
    Type: AWS::Lambda::Function
    Properties:
      Description: Copies objects from a source S3 bucket to a destination
      Handler: index.handler
      Runtime: python2.7
      Role: !GetAtt AWSSBInjectedCopyZipsRole.Arn
      Timeout: 240
      Code:
        ZipFile: |
          import json
          import logging
          import threading
          import boto3
          import cfnresponse

          def copy_objects(source_bucket, dest_bucket, prefix, objects):
              s3 = boto3.client('s3')
              for o in objects:
                  key = prefix + o
                  copy_source = {
                      'Bucket': source_bucket,
                      'Key': key
                  }
                  print('copy_source: %s' % copy_source)
                  print('dest_bucket = %s' % dest_bucket)
                  print('key = %s' % key)
                  s3.copy_object(CopySource=copy_source, Bucket=dest_bucket, Key=key)

          def delete_objects(bucket, prefix, objects):
              s3 = boto3.client('s3')
              objects = {'Objects': [{'Key': prefix + o} for o in objects]}
              s3.delete_objects(Bucket=bucket, Delete=objects)

          def timeout(event, context):
              logging.error('Execution is about to time out, sending failure response to CloudFormation')
              cfnresponse.send(event, context, cfnresponse.FAILED, {}, None)

          def handler(event, context):
              timer = threading.Timer((context.get_remaining_time_in_millis() / 1000.00) - 0.5, timeout, args=[event, context])
              timer.start()
              print('Received event: %s' % json.dumps(event))
              status = cfnresponse.SUCCESS
              try:
                  source_bucket = event['ResourceProperties']['SourceBucket']
                  dest_bucket = event['ResourceProperties']['DestBucket']
                  prefix = event['ResourceProperties']['Prefix']
                  objects = event['ResourceProperties']['Objects']
                  if event['RequestType'] == 'Delete':
                      delete_objects(dest_bucket, prefix, objects)
                  else:
                      copy_objects(source_bucket, dest_bucket, prefix, objects)
              except Exception as e:
                  logging.error('Exception: %s' % e, exc_info=True)
                  status = cfnresponse.FAILED
              finally:
                  timer.cancel()
                  cfnresponse.send(event, context, status, {}, None)
  AWSSBInjectedGetCidrsRole:
    Condition: AutoCidrs
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: /
      Policies:
      - PolicyName: cfn_utility_get_cidrs
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
            - ec2:DescribeVpcs
            - ec2:DescribeSubnets
            - ec2:DescribeAvailabilityZones
            - logs:CreateLogGroup
            - logs:CreateLogStream
            - logs:PutLogEvents
            Resource: '*'
  AWSSBInjectedGetCidrsLambda:
    DependsOn: AWSSBInjectedCopyZips
    Condition: AutoCidrs
    Type: AWS::Lambda::Function
    Properties:
      Handler: lambda_function.handler
      Role:
        Fn::GetAtt:
        - AWSSBInjectedGetCidrsRole
        - Arn
      Code:
        S3Bucket: !Ref AWSSBInjectedLambdaZipsBucket
        S3Key: functions/get_cidrs/lambda_function.zip
      Runtime: python2.7
      Timeout: '60'
  AWSSBInjectedGetCidrs:
    Condition: AutoCidrs
    Type: AWS::CloudFormation::CustomResource
    Properties:
      ServiceToken: !GetAtt AWSSBInjectedGetCidrsLambda.Arn
      Qty: 1
      VpcId: !Ref VpcId
      CidrSize: !Ref CidrSize
Outputs:
  EndpointAddress:
    Value:
      !GetAtt
      - ElastiCacheCluster
      - ConfigurationEndpoint.Address

Update: At least it's a valid CF template now. However it still won't be cached by the broker???

johannesvietze commented 4 years ago

any update on this? will there be an official template?

saurabh-devops commented 4 years ago

Anyupdate on this. I am trying to figure out to it as add custom service. But cant understand the process

kferrone commented 4 years ago

Here is my final Redis Broker Template. It works well and if you know Cloudformation it should not be too difficult to edit to your own liking.

AWSTemplateFormatVersion: 2010-09-09
Description: >-
  Stands up an ElastiCache Replication Group Multi-AZ in a
  particular VPC specifying then a Subnet Group and a Security Group, VpcId is not directly
  referenced by the Replication Group resource, and is instead inferred by the Security
  Group and Subnet Group.

Metadata:
  AWS::ServiceBroker::Specification:
    Version: 1.0
    Tags:
    - AWS
    - elasticache
    - redis
    Name: redis
    DisplayName: Amazon ElastiCache Redis
    LongDescription: Amazon ElastiCache is a web service that makes it easy to set
      up, manage, and scale distributed in-memory cache environments in the cloud.
      It provides a high performance, resizeable, and cost-effective in-memory cache,
      while removing the complexity associated with deploying and managing a distributed
      cache environment.
    ImageUrl: https://s3.amazonaws.com/awsservicebroker/icons/Database_AmazonElasticCache_LARGE.png
    DocumentationUrl: https://aws.amazon.com/documentation/elasticache/
    ProviderDisplayName: Amazon Web Services
    ServicePlans:
      dev:
        DisplayName: Development
        Description: Simple single redis
        LongDescription: >-
          Creates a simple redis with one node in one availability zone and not
          using cluster mode enabled. Also no encryption for ease of use.
        Cost: https://aws.amazon.com/elasticache/pricing/
        ParameterValues:
          NumberOfAvailabilityZones: 1
          NumNodeGroups: 1
          NumCacheClusters: 1
          RedisParams: ""
          AuthToken: Auto
          AtRestEncryptionEnabled: 'false'
      production:
        DisplayName: Production
        Description: Production ready redis
        LongDescription: >-
          Creates a production ready redis cluster with encryption enabled and auto password.
          It is still up to the deployer to define how scaled it should be, i.e. you still must
          set NumNodeGroups and NumCacheClusters.
        Cost: https://aws.amazon.com/elasticache/pricing/
        ParameterValues:
          AuthToken: Auto
          AtRestEncryptionEnabled: 'true'
      custom:
        DisplayName: Custom
        Description: Custom Configuration for Advanced deployments
        LongDescription: Creates an Amazon ElastiCache for redis with custom configuration
        Cost: https://aws.amazon.com/elasticache/pricing/
        ParameterValues: {}
  AWS::CloudFormation::Interface:
    ParameterGroups:
    - Label:
        default: Redis Settings
      Parameters:
      - CacheNodeType
      - EngineVersion
      - NumCacheClusters
      - NumNodeGroups
      - RedisPort
      - ReplicationGroupDescription
      - AuthToken
      - AtRestEncryptionEnabled
      - RedisParams
    - Label:
        default: Network and Security
      Parameters:
      - VpcId
      - AccessCidr
      - SubnetList
      - SubnetGroup
      - NumberOfAvailabilityZones
    - Label:
        default: Maintenance
      Parameters:
      - SnapshotRetentionLimit
      - SnapshotWindow
      - PreferredMaintenanceWindow
    ParameterLabels:
      CacheNodeType:
        default: Cache Node Type
      EngineVersion:
        default: Redis Version
      NumberOfAvailabilityZones:
        default: Number Of Availability Zones
      NumCacheClusters:
        default: Number of Clusters
      NumNodeGroups:
        default: How Many Node Groups
      RedisPort:
        default: Redis Port
      ReplicationGroupDescription:
        default: Replication Groups Description
      VpcId:
        default: VPC ID
      SubnetList:
        default: List of Subnets
      SubnetGroup:
        default: Subnet Group Name
      AccessCidr:
        default: CIDR IP
      SnapshotRetentionLimit:
        default: Snapshot retention Limit
      SnapshotWindow:
        default: Snapshot Window
      PreferredMaintenanceWindow:
        default: Preferred MaintenanceWindow
      AuthToken:
        default: Authorization Token
      AtRestEncryptionEnabled:
        default: Enable At Rest Encryption
      RedisParams:
        default: Custom Redis Parameters

Parameters:
  CacheNodeType:
    Description: The instance type the nodes will launch under.
    Type: String
    Default: cache.m3.medium
#    AllowedValues:
#      - cache.t2.micro
#      - cache.t2.small
#      - cache.t2.medium
#      - cache.m3.medium
#      - cache.m3.large
#      - cache.m3.xlarge
#      - cache.m3.2xlarge
#      - cache.r3.large
#      - cache.r3.xlarge
#      - cache.r3.2xlarge
#      - cache.r3.4xlarge
#      - cache.r3.8xlarge
  EngineVersion:
    Description: Choose the version of Redis to use
    Type: String
    Default: "5.0.6"
    AllowedValues:
      - "5.0.6"
      - "5.0.5"
      - "5.0.4"
      - "5.0.3"
      - "5.0.0"
      - "4.0.10"
      - "3.2.10"
      - "3.2.6"
      - "3.2.4"
      - "2.8.24"
      - "2.8.23"
      - "2.8.22"
      - "2.8.21"
      - "2.8.19"
      - "2.8.6"
      - "2.6.13"
  NumberOfAvailabilityZones:
    Description: Quantity of subnets to use, if selecting more than 2 the region this
      stack is in must have at least that many Availability Zones
    Type: String
    Default: '2'
    AllowedValues:
    - '1'
    - '2'
    - '3'
    - '4'
    - '5'
  NumCacheClusters:
    Description: >-
      The number of cache clusters for this replication group. If MultiAZ
      support is enabled, you must specify a value greater than 1.
    Default: '2'
    Type: Number
    MinValue: '1'
    MaxValue: '6'
  NumNodeGroups:
    Description: >-
      The number of NodeGroups to create. If this value is more than one, then the
      NumCacheClusters parameter represents the ReplicasPerNodeGroup for each of
      the NodeGroups created.
    Default: '1'
    Type: String
  RedisPort:
    Description: >-
      The port number on which each member of the replication group accepts
      connections.
    Type: Number
    Default: '6379'
    MinValue: '1'
    MaxValue: '65535'
  ReplicationGroupDescription:
    Description: The description of the replication group.
    Type: String
    Default: Redis replication group
  VpcId:
    Description: The VPC to create this ReplicationGroup under
    Type: 'AWS::EC2::VPC::Id'
  AccessCidr:
    Description: The CIDR you want to access to the Replication Group
    Type: String
    Default: 0.0.0.0/0
    AllowedPattern: '(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2})'
    MinLength: '9'
    MaxLength: '18'
    ConstraintDescription: must be a valid IP CIDR range of the form x.x.x.x/x
  SnapshotRetentionLimit:
    Description: >-
      The number of days that ElastiCache retains automatic snapshots before
      deleting them.
    Type: Number
    Default: '7'
  SnapshotWindow:
    Description: >-
      The time range (in UTC) when ElastiCache takes a daily snapshot of your node group.
    Type: String
    Default: '05:00-09:00'
    AllowedPattern: '\d{2}:\d{2}-\d{2}:\d{2}'
    ConstraintDescription: 'must be a valid timestamp range, for example 05:00-09:00'
  PreferredMaintenanceWindow:
    Description: >-
      The weekly time range during which system maintenance can occur. Use the following
      format to specify a time range; ddd:hh24:mi-ddd:hh24:mi (24H Clock UTC).
    Type: String
    Default: 'sun:22:00-sun:23:30'
    AllowedPattern: >-
      (mon|tue|wed|thu|fri|sat|sun):\d{2}:\d{2}-(mon|tue|wed|thu|fri|sat|sun):\d{2}:\d{2}
    ConstraintDescription: >-
      must be a valid timestamp range with day of week, for example
      sun:22:00-sun:23:30
  AtRestEncryptionEnabled:
    Description: A flag that enables encryption at rest when set to true.
    Default: 'false'
    Type: String
    AllowedValues:
    - 'true'
    - 'false'
  AuthToken:
    Description: A password to access redis
    Type: String
    Default: Auto
    NoEcho: 'true'
    #MinLength: 16
    #MaxLength: 128
    #AllowedPattern: '^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[!&#$^<>-])(?!.*[{}()*/+=%@_:;''""?\ ]).{16,128}$'
    #ConstraintDescription: Make a better password
  SubnetList:
    Description: Pass a CSV list of subnets.
    Type: String
    Default: ""
  SubnetGroup:
    Description: The group name of the subnets to query for.
    Type: String
    Default: Isolated
  RedisParams:
    Description: >-
      Stringified json to override the default properties of the parameter group. When cluster mode
      the param group does extend the cluster mode param group. Otherwise just hte basic param group.
    Type: String
    Default: ""
  GeneratePasswordLambda:
    Description: The arn of the lambda which generates passwords
    Type: String
  GetSubnetGroupLambda:
    Description: The arn of hte lambda which gets subnet groups
    Type: String

Mappings:
  Version2Family:
    5.0.6:
      Family: redis5.0
    5.0.5:
      Family: redis5.0
    5.0.4:
      Family: redis5.0
    5.0.3:
      Family: redis5.0
    5.0.0:
      Family: redis5.0
    4.0.10:
      Family: redis4.0
    3.2.10:
      Family: redis3.2
    3.2.6:
      Family: redis3.2
    3.2.4:
      Family: redis3.2
    2.8.24:
      Family: redis2.8
    2.8.23:
      Family: redis2.8
    2.8.22:
      Family: redis2.8
    2.8.21:
      Family: redis2.8
    2.8.19:
      Family: redis2.8
    2.8.6:
      Family: redis2.8
    2.6.13:
      Family: redis2.6

Conditions:

  AutoToken:
    !Equals
    - !Ref AuthToken
    - Auto

  NoToken:
    !Equals
    - !Ref AuthToken
    - ""

  UseToken:
    !Or
    - Condition: AutoToken
    - !Not
      - Condition: NoToken

  NotCustomSubnetList:
    !Equals
    - !Ref SubnetList
    - ""

  CustomSubnetList:
    !Not
    - Condition: NotCustomSubnetList

  EncryptionEnabled:
    !Or
    - !Equals
      - !Ref AtRestEncryptionEnabled
      - 'true'
    - Condition: UseToken

  MultiAZ:
    !Not
    - !Equals
      - !Ref NumberOfAvailabilityZones
      - 1

  ClusterMode:
    !Not
    - !Equals
      - !Ref NumNodeGroups
      - 1

  UsingCustomParams:
    !Not
    - !Equals
      - !Ref RedisParams
      - ""

Resources:

  ReplicationGroup:
    Type: 'AWS::ElastiCache::ReplicationGroup'
    Properties:
      Engine: redis
      EngineVersion: !Ref EngineVersion
      AutomaticFailoverEnabled:
        !If
        - MultiAZ
        - true
        - false
      CacheNodeType: !Ref CacheNodeType
      CacheSubnetGroupName: !Ref ElastiCacheSubnetGroup
      CacheParameterGroupName:
        !If
        - UsingCustomParams
        - !Ref ElasticacheParameterGroup
        - !Sub
          - "default.${Family}${ClusterOn}"
          - Family:
              !FindInMap
              - Version2Family
              - !Ref EngineVersion
              - Family
            ClusterOn:
              !If
              - ClusterMode
              - ".cluster.on"
              - ""
      NumCacheClusters:
        !If
        - ClusterMode
        - !Ref 'AWS::NoValue'
        - !If
          - MultiAZ
          - !Ref NumCacheClusters
          - 1
      NumNodeGroups:
        !If
        - ClusterMode
        - !Ref NumNodeGroups
        - !Ref 'AWS::NoValue'
      ReplicasPerNodeGroup:
        !If
        - ClusterMode
        - !Ref NumCacheClusters
        - !Ref 'AWS::NoValue'
      Port: !Ref RedisPort
      PreferredMaintenanceWindow: !Ref PreferredMaintenanceWindow
      ReplicationGroupDescription: !Ref ReplicationGroupDescription
      SecurityGroupIds:
        - !GetAtt
          - SecurityGroup
          - GroupId
      SnapshotRetentionLimit: !Ref SnapshotRetentionLimit
      SnapshotWindow: !Ref SnapshotWindow
      AtRestEncryptionEnabled: !Ref AtRestEncryptionEnabled
      TransitEncryptionEnabled:
        !If
        - UseToken
        - true
        - false
      AuthToken:
        !If
        - AutoToken
        - !GetAtt GeneratedPassword.MasterUserPassword
        - !If
          - NoToken
          - !Ref 'AWS::NoValue'
          - !Ref AuthToken
      KmsKeyId:
        !If
        - EncryptionEnabled
        - !GetAtt KMSKey.Arn
        - !Ref AWS::NoValue

  ElasticacheParameterGroup:
    Type: AWS::ElastiCache::ParameterGroup
    Condition: UsingCustomParams
    Properties:
      CacheParameterGroupFamily:
        !FindInMap
        - Version2Family
        - !Ref EngineVersion
        - Family
      Description: Customizable group for the redis broker
      Properties: !Ref RedisParams

  ElastiCacheSubnetGroup:
    Type: 'AWS::ElastiCache::SubnetGroup'
    Properties:
      Description: Subnet Group for the Replication Group
      SubnetIds:
        !If
        - CustomSubnetList
        - !Split [',', !Ref SubnetList]
        - !Split [',', !GetAtt SubnetGroupList.SubnetIds]

  SecurityGroup:
    Type: 'AWS::EC2::SecurityGroup'
    Properties:
      GroupDescription: Security Group for Replication Group
      VpcId: !Ref VpcId
      SecurityGroupIngress:
        - CidrIp: !Ref AccessCidr
          FromPort: !Ref RedisPort
          ToPort: !Ref RedisPort
          IpProtocol: tcp
      SecurityGroupEgress:
        - IpProtocol: '-1'
          FromPort: '-1'
          ToPort: '-1'
          CidrIp: 0.0.0.0/0

  KMSKey:
    Condition: EncryptionEnabled
    Type: AWS::KMS::Key
    Properties:
      Description: KMS Key for Broker Redis
      Enabled: 'true'
      EnableKeyRotation: 'true'
      KeyPolicy:
        Version: '2012-10-17'
        Id: key-default-1
        Statement:
        - Sid: Allow administration of the key
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:root
          Action:
          - kms:Create*
          - kms:Describe*
          - kms:Enable*
          - kms:List*
          - kms:Put*
          - kms:Update*
          - kms:Revoke*
          - kms:Disable*
          - kms:Get*
          - kms:Delete*
          - kms:ScheduleKeyDeletion
          - kms:CancelKeyDeletion
          Resource: '*'

  SubnetGroupList:
    Type: Custom::SubnetGroup
    Condition: NotCustomSubnetList
    Properties:
      ServiceToken: !Ref GetSubnetGroupLambda
      VpcId: !Ref VpcId
      GroupName: !Ref SubnetGroup
      Qty: !Ref NumberOfAvailabilityZones

  GeneratedPassword:
    Type: AWS::CloudFormation::CustomResource
    Condition: AutoToken
    Properties:
      ServiceToken: !Ref GeneratePasswordLambda
      Length: 32
      SpecialCharachters: '!&#$^<>-'

Outputs:
  RedisHost:
    Description: The host for apps to connect to the replication group.
    Value: !GetAtt
      - ReplicationGroup
      - PrimaryEndPoint.Address
  RedisPort:
    Description: The port for apps to connect to the replication group.
    Value:
      !Sub
      - '${PortNum}'
      - PortNum:
          !GetAtt
            - ReplicationGroup
            - PrimaryEndPoint.Port
#  RedisToken:
#    Description: The auth token to connect to redis instance.
#    Value:
#      !If
#      - AutoToken
#      - !GetAtt GeneratedPassword.MasterUserPassword
#      - !If
#        - NoToken
#        - !Ref 'AWS::NoValue'
#        - !Ref AuthToken
  RedisEndpoint:
    Description: The primary endpoint location
    Value: !Join
      - ''
      - - 'redis://'
        - !GetAtt
          - ReplicationGroup
          - PrimaryEndPoint.Address
        - ':'
        - !GetAtt
          - ReplicationGroup
          - PrimaryEndPoint.Port
saurabh-devops commented 4 years ago

Thanks for your help..but I also need help on how to add it to the awsservicebroker..I cant find the awsservicebroker bucket in my AWS account. Also the documentation on how to add service to AWS service broker is not clear, at least to me.