RDS VPC Read Replica with different subnet group

[hashibot]

This issue was originally opened by @phoolish as hashicorp/terraform#11919. It was migrated here as part of the provider split. The original body of the issue is below.

---

I've gotten this to work from the AWS console, but I can't do it from terraform.

Terraform Version

v0.8.5

Affected Resource(s)

• aws_db_instance

Terraform Configuration Files

resource "aws_db_instance" "prod_rds_rep" {
  identifier = "prod-rds-rep"
  replicate_source_db = "${aws_db_instance.prod_rds.id}"
  instance_class = "db.t2.small"
  backup_retention_period = 0
  publicly_accessible = true

  apply_immediately = true
  auto_minor_version_upgrade = true

  db_subnet_group_name = "${aws_db_subnet_group.prod_rds_rep_sg.id}"
}

Expected Behavior

Should create an RDS instance in a different subnet group from the source db.

Actual Behavior

1 error(s) occurred:

* aws_db_instance.prod_rds_rep: Error creating DB Instance: DBSubnetGroupNotAllowedFault: DbSubnetGroupName should not be specified for read replicas that are created in the same region as the master
        status code: 400, request id:

[taisyo7333]

I have the same problem.

$ terraform --version
Terraform v0.11.0
+ provider.aws v1.3.1

[albrechtsimon]

This is an AWS limitation. Check the docs for --db-subnet-group-name (string)

I am facing this issue as well.

[jonathortense]

@phoolish, did you manage to work around this? I have the same problem here. If I can do this through the web console, why not via API?

[phoolish]

@jonathortense sorry for the late reply, but I was on an extended vacation. If I remember correctly, we could use the AWS console to make this work and then import the resource into terraform.

[maxcountryman]

How are you able to do this from the web console? I've been trying to find where this is possible without success...

[errriclee]

https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBInstanceReadReplica.html

Indeed it is not allowed by the API now (see DBSubnetGroupName and DBSubnetGroupNotAllowedFault). This worked last time I tried it, which was 9/13/2017.

[maxcountryman]

That’s unfortunate. I don’t reallt understand the limitation. Enterprise support gave me a similar answer.

[errriclee]

To do this in the console:

1. Go to source DB
2. Actions -> Create Read Replica
3. Change Destination DB subnet group
4. Create

[maxcountryman]

@errriclee I don’t see those options for an Aurora instance.

[matanlb]

I had the same issue working with PostgresSql databases. Turns out it a mistake in the aws-cli (and carried on to terraform docs)

Under replicate_source_db you need pass the rds instance arn NOT identifier. I've been talking to AWS support and it works, I've created a read-replica via terraform.

EDIT NOTE After the initial creation of the replica you'll need to change the replicate_source_db to the source identifier otherwise terraform will try to update it to the arn.

[ktham]

@matanlb you were able to create a read replica that's on a different db subnet group? I used the arn and it still gives the same error as the OP

[errriclee]

I can confirm that you can create replicas in another subnet group using the ARN, at least via API.

Here's a quick test I did in Ruby:

require 'aws-sdk'
require 'pp'

client = Aws::RDS::Client.new
source_db = "arn:aws:rds:us-west-2:123456789012:db:my-instance"
source_db_subnet_group = "my-subnets"
target_db_subnet_group =  "other-subnets"

resp = client.create_db_instance_read_replica({
  copy_tags_to_snapshot: true,
  db_instance_class: "db.m4.large",
  db_subnet_group_name: target_db_subnet_group,
  db_instance_identifier: "mytest",
  publicly_accessible: false,
  source_db_instance_identifier: source_db,
  storage_type: "gp2"
})

pp resp

I have not tried with Terraform yet, but I was getting the same error as I received earlier from Terraform when I only specified the identifier, not the ARN. I was given this resolution by AWS Support.

[matanlb]

@ktham yes, in my case I've created a read replica in a different subnet on the same VPC.

[mconigliaro]

A lot of confusing comments in here... My understanding from the documentation is that you can only create a read replica in a different subnet group from your source database if the RDS instances are in different regions. Is the documentation wrong?

[matanlb]

@mconigliaro yes, the docs are wrong on that one (or maybe just out of date). It is definitely possible to create a read replica in the same region (even the same VPC) but in a different subnet.

Like I've mentioned before this is an error that stamed from wrong documentation of AWS. I've talked to their support and they instructed me to replace the identifier with the ARN. They were referring to the CLI commands but I in terraform as well.

[mconigliaro]

OK, I've confirmed what @matanlb said above. To create a PostgreSQL instance in a different subnet group, but in the same region as the source database, I had to set replicate_source_db to an ARN during creation. Interestingly, I've noticed that this always results in the replica having the same security groups as the source database. But then if I set replicate_source_db back to the id (which you need to do anyway) and run terraform apply again, it fixes the security groups.

Pretty annoying bug(s), but at least we have workarounds.

[Morsicus]

Hey,

I cannot understand why we need to switch back replicate_source_db to the ID?

Where is the limitation to use ARN rather than ID?

[orimarti]

Changing ID to ARN works for me, but in my case, replicating from EC2 classic gives this error now: At least one security group '****' (Non-VPC) and subnet group '****' (in VPC '**') are not in common VPC. Although I'm assigning it a security group from the VPC looks like it's assigning the same one on the classic instance. Any workaround if I want to do from EC2 classic to VPC?

[rosenbergj]

I want to note that the Terraform documentation still seems to be out of date. I tried creating a replica of an RDS database in the same account/region/VPC, but different subnet group. Specifying an identifier for replicate_source_db resulted in a DBSubnetGroupNotAllowedFault error, despite that being what the docs say at https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance#replicate_source_db . Specifying an arn worked, which I only knew to do by finding this issue.

I also had to change it from arn to identifier after creating the database. If I didn't, terraform state thought that I was changing the source DB to a different one, and threw an error.

[anoshi]

I tried creating a replica of an RDS database in the same account/region/VPC, but different subnet group. Specifying an identifier for replicate_source_db resulted in a DBSubnetGroupNotAllowedFault error

If you've found a way to make this work (replica uses a different subnet group), it would be purely by chance/luck :-D

Terraform doco hints at the root cause here:

When working with read replicas, it should be specified only if the source database specifies an instance in another AWS Region

This is stated explicitly in the AWS Doco:

DBSubnetGroupName ... Can only be specified if the source DB instance identifier specifies a DB instance in another AWS Region.

[ajaymahore5517]

I have similar issue but its with vpc_security_group_ids. I am creating a primary RDS(MySQL) and a cross region read replica using terraform, while passing 'vpc_security_group_ids' this in read replica, I am getting error like it can be defined only once. When i do not pass this parameter it gives me an error as same security group IDs will not be available in other region. Any suggestion to mitigate this issue?

[ajaymahore5517]

I want to note that the Terraform documentation still seems to be out of date. I tried creating a replica of an RDS database in the same account/region/VPC, but different subnet group. Specifying an identifier for replicate_source_db resulted in a DBSubnetGroupNotAllowedFault error, despite that being what the docs say at https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance#replicate_source_db . Specifying an arn worked, which I only knew to do by finding this issue.

I also had to change it from arn to identifier after creating the database. If I didn't, terraform state thought that I was changing the source DB to a different one, and threw an error.

ARN is required if you are creating replica in other region, identifier should be good enough for replicas in same region.

[pmalafosse]

+1 @rosenbergj Terraform documentation should be updated like there (see 'Constraints') https://docs.aws.amazon.com/cli/latest/reference/rds/create-db-instance-read-replica.html and we shouldn't need to change the ARN to the ID once the read-replica has been created to avoid errors

[gvirtuoso]

Update from Dec 20 / 2021.
Still not working.
You can create a read replica in a different subnet group using the AWS Web Console, but with Terraform still is a mess.

[zytek]

Update from Dec 20 / 2021. Still not working. You can create a read replica in a different subnet group using the AWS Web Console, but with Terraform still is a mess.

You can create it via terraform if you specified ARN instead of IDENTIFIER as shown here: replicate_source_db = aws_db_instance.this.arn

[banagevikas]

Even after specifying ARN I still get the same issue.

[ivankovnatsky]

In my case I had to play around a little bit.

First run:

1. Change to arn for replicate_source_db
2. Disable multi AZ

Second run:

1. Change back to id for replicate_source_db
2. Enable back multi AZ

[dialloaliou]

This worked for me:

1. When applying for the first time, use arn.
2. On the second run you must replace arn with id, otherwise on every terraform apply it will be as if you want to modify the source db as below:

~ replicate_source_db = "example-read-replica" -> "arn:aws:rds:<<aws_region>>:<<aws_acount_id>>:db:db-example-read-replica"

[ARolek]

We just went through this same journey. I submitted a ticket to AWS via the docs.

[gvasileiou]

For solving arn --> id for the second apply you can just use lifecycle manager and replica instance.

lifecycle {
    ignore_changes = [replicate_source_db]
  }