Redis cluster reports SubnetGroup does not exist, if building SubnetGroup part of the same stack

[cristianrat]

I am building a VPC, with private, public and protected subnets. Part of the same stack, I need to create a Redis cluster (and some other things later, but those don't matter for now). It requires a SubnetGroup. While building the stack, including the SubnetGroup, all is well and it builds.

When moving to the Redis cluster, it reports the SubnetGroup does not exist (technically, it doesn't exist yet, but that's what the stack is doing).

A work around is to run the stack in stages, go as far as subnet group, then uncomment the rest and build redis cluster and so on. Not a workable solution in reality. It's a bug as far as I can tell, but I would be happy to know if I'm doing the below wrong.

Reproduction Steps

Simply build the stack

from aws_cdk import (
    aws_ec2 as ec2,
    aws_elasticache as elasticache,
    aws_ecs as ecs,
    core
)

class IntegrationsVPC(core.Stack):
    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        # Create new VPC
        vpc = ec2.Vpc(
            self, "Default",
            max_azs=3,
            nat_gateways=1,
            cidr=ec2.Vpc.DEFAULT_CIDR_RANGE,
            subnet_configuration=[
                ec2.SubnetConfiguration(
                    name="Private-Subnet",
                    subnet_type=ec2.SubnetType.PRIVATE,
                    cidr_mask=19,
                    reserved=None
                ),
                ec2.SubnetConfiguration(
                    name="Public-Subnet",
                    subnet_type=ec2.SubnetType.PUBLIC,
                    cidr_mask=22,
                    reserved=None
                ),
                ec2.SubnetConfiguration(
                    name="Isolated-Subnet",
                    subnet_type=ec2.SubnetType.ISOLATED,
                    cidr_mask=28,
                    reserved=None
                )
            ]
        )

        # Try subnet group
        subnet_group = elasticache.CfnSubnetGroup(
            scope=self,
            id="Testing-Subnet-Group",
            description="Group private subnets for redis access.",
            subnet_ids=[subnet.subnet_id for subnet in vpc.private_subnets],
            cache_subnet_group_name="test-int-private-subnets"
        )
        redis_security_group = ec2.SecurityGroup(
            scope=self,
            id="TEMP-redis-SG",
            vpc=vpc,
            allow_all_outbound=False
        )

        redis_cluster = elasticache.CfnCacheCluster(
            scope=self,
            cache_node_type="cache.t2.micro",
            id="testmy-redis",
            engine="redis",
            num_cache_nodes=1,
            vpc_security_group_ids=[redis_security_group.security_group_id],
            cache_subnet_group_name=subnet_group.cache_subnet_group_name,
            cluster_name="testmy-redis"
        )

app = core.App()
IntegrationsVPC(app, "Integrations-VPC-TEMP", env={
    'account': 'XXXXXXXXXX',
    'region': 'eu-west-2' # or what ever you want
  })
app.synth()

Error Log

15/43 | 1:09:41 PM | CREATE_FAILED        | AWS::ElastiCache::CacheCluster        | integrations-redis (integrationsredis) Cache Subnet Group test-int-private-subnets does not exist. (Service: AmazonElastiCache; Status Code: 400; Error Code: CacheSubnetGroupNotFoundFault; Request ID: 475a20d2-fc40-4d94-809c-******)

Environment

• CLI Version : 1.12.0 (build 923055e)
• Framework Version: 1.12.0
• OS : Ubuntu 18
• Language : Python

Other

---

This is :bug: Bug Report

[NGL321]

Hi @cristianrat,

Thank you for reporting! Is it possible this is related to #3098?

[cristianrat]

@NGL321 Don't think so, as they get the wrong VPC message, where I am getting something else. Creating the subnet group would also work, if I run the stack only to that stage.

[skinny85]

@cristianrat thanks for opening the issue. I believe what you want is to add a dependency between subnet_group and redis_cluster, so that CFN knows to deploy the former before the latter:

redis_cluster.add_depends_on(subnet_group)

Or you can do this:

        redis_cluster = elasticache.CfnCacheCluster(
            scope=self,
            cache_subnet_group_name=subnet_group.ref,
            # rest of the properties as above...
        )

which I believe is equivalent to what you had originally, and adds the dependency automatically.

Let me know if this helps!

Thanks, Adam

[cristianrat]

Thanks for the tip! Wish I had known about it sooner. Will try it tomorrow morning, first thing and let you know if works.

On Wed, 9 Oct 2019, 21:26 Adam Ruka, notifications@github.com wrote:

@cristianrat https://github.com/cristianrat thanks for opening the issue. I believe what you want is to add a dependency between subnet_group and redis_cluster, so that CFN knows to deploy the former before the latter:

redis_cluster.add_depends_on();

Or you can do this:

    redis_cluster = elasticache.CfnCacheCluster(
        scope=self,
        cache_subnet_group_name=subnet_group.ref,
        # rest of the properties as above...
    )

which I believe is equivalent to what you had originally, and adds the dependency automatically.

Let me know if this helps!

Thanks, Adam

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/aws/aws-cdk/issues/4411?email_source=notifications&email_token=ALFYCRBQP2LA6CB2AYQEQJ3QNY46ZA5CNFSM4I6RT6M2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAZHJFY#issuecomment-540177559, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALFYCRDACQRYMG4TVEAMUD3QNY46ZANCNFSM4I6RT6MQ .

[cristianrat]

Thank you @skinny85 It works great! I am happy this is not a bug :)

[skinny85]

Me too :D

[namedgraph]

I hit a related issue which turned out to be #6519

[crucialfelix]

For cdk 2 this is:

    const subnetGroupName = `SubnetGroup-${name}`;
    // This needs to be created first. It is identified by "cacheSubnetGroupName"
    // and then found and used by the CfnCacheCluster.
    const subnetGroup = new CfnSubnetGroup(this, subnetGroupName, {
      description: `List of subnets used for ${id}`,
      subnetIds: vpc.privateSubnets.map((subnet) => subnet.subnetId),
      cacheSubnetGroupName: subnetGroupName
    });

    const cache = new CfnCacheCluster(this, name, {
      clusterName: name,
      engine: "memcached",
      engineVersion: "1.6.17",
      cacheNodeType: "cache.t2.micro",
      numCacheNodes: 1,
      azMode: "single-az",
      // preferredAvailabilityZones: ["us-east-1b"],
      autoMinorVersionUpgrade: true,
      vpcSecurityGroupIds: [sg.securityGroupId],
      /**
       * The name of the subnet group to be used for the cluster.
       *
       * Use this parameter only when you are creating a cluster in an Amazon Virtual Private Cloud (Amazon VPC).
       *
       * > If you're going to launch your cluster in an Amazon VPC, you need to create a subnet group before
       * you start creating a cluster. For more information,
       * see [AWS::ElastiCache::SubnetGroup](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticache-subnetgroup.html) .
       */
      cacheSubnetGroupName: subnetGroup.cacheSubnetGroupName
    });

    // Add a dependency so that the subnet group is created before the cache cluster.
    cache.addDependency(subnetGroup);