Manage Route53 TXT record values as independent resources

beanaroo commented 4 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Terraform can currently only manage entire DNS records
Amazon Route53 does not permit multiple TXT records for a domain

This poses a problem when sharing a domain across states (accounts/environments/stages) for things like ACM (SSL) and SES (Email).

For example, AWS Workmail will create a TXT record for _amazonses.acme.com.

Terraforming an SES domain identity along with validation fails because the above record already exists. The same is true when adding any additional SES domain identity for the same domain in another region and/or account.

Since Amazon does not permit multiple TXT records for a domain, it would be ideal if Terraform could manage the values independently.

New or Affected Resource(s)

data.aws_route53_txt_record - provides a way to identify an external TXT record
aws_route53_txt_record_value - a value to be added to the existing set of values

Potential Terraform Configuration


data "aws_route53_txt_record" "amazon-ses" {
  zone_id = "${aws_route53_zone.primary.zone_id}"
  name    = "_amazonses.acme.com."
}

resource "aws_route53_txt_record_value" "acme-ses-sydney" {
  record = "${data.aws_route53_txt_record.amazon-ses.id}"
  values = ["${aws_ses_domain_identity.acme-sydney.verification_token}"]
}

Alternatives

If someone can provide alternative solutions to sharing a TXT record across states/accounts/regions/stages/environments, recommendations are welcome.

jnewton03 commented 3 years ago

@beanaroo did you ever find a workaround or solution to this problem? I'm having the same issue trying to implement SES across multiple accounts.

beanaroo commented 3 years ago

@jnewton03 Unfortunately no solution but a less than ideal workaround...

We've moved all SES and ACM resources into our Route53 module that's used by a single state. This now requires some double-handling when introducing/removing resources and introduced tight coupling between our environment states and the common resources account state that hosts Route53.

We'd still benefit from the above capability.

Our configuration looks like this

```yaml domains: acme: # Creates a hosted zone name: 'acme.com' provision: true delegated: false # Explicit records records: - name: foo.acme.com. type: A ttl: 3600 value: - 333.101.123.99 # SES identities in relevant accounts along with their aggregated verification records in the hosted zone account ses: workmail: txt: 'r/zsdfasdgxcvgbsdfgertasdfgsdfgsdfgs=' notifications: true accounts: # Shared Stuff - account: '34562345236' regions: [us-east-1] # Dev - account: '45236345623' regions: [ap-southeast-2, eu-west-1] # Staging - account: '52365462334' regions: [ap-southeast-2, eu-west-1] # Production - account: '36546233452' regions: [ap-southeast-2, eu-west-1] # ACM Certificates in relevant accounts along with their aggregated verification records in the hosted zone account acm: alternatives: - '*.acme.com' accounts: # Dev - account: '45236345623' regions: [ap-southeast-2, us-east-1] # Staging - account: '52365462334' regions: [us-east-1] # Production - account: '36546233452' regions: [us-east-1] ```

mousedownmike commented 3 years ago

@jnewton03 After struggling with this for a long time I ended up with the following solution:

# Create the Verification Token
resource aws_ses_domain_identity example_com {
  domain   = "example.com"
}

# Read the existing SES TXT record from DNS
data dns_txt_record_set ses_txt {
  host = "_amazonses.example.com"
}

# Write (or overwrite) the verification TXT record adding the new record to the
# existing records (if it isn't already there)
resource aws_route53_record ses_txt {
  allow_overwrite = true
  zone_id         = data.aws_route53_zone.example_com.id
  name            = "_amazonses.example.com"
  ttl             = 600
  type            = "TXT"
  records         = toset(concat(data.dns_txt_record_set.ses_txt.records, [
    aws_ses_domain_identity.example_com.verification_token]))
  lifecycle {
    prevent_destroy = true
  }
}

The trick is that allow_overwrite = true will allow multiple Terraform workspaces to write to the "same" record. The prevent_destroy = true option prevents the record from being deleted. This is also the biggest problem, tokens are never removed. If SES instances are being brought up and torn down regularly, that might be a problem.

I would love to hear if anyone can find other drawbacks to this approach. It's definitely a hack but it's all I can come up with.

I would also like to know why Amazon's SES verification records can't be like the DKIM records where the name of the TXT record includes a unique token.

syquus commented 3 years ago

@mousedownmike that's a nice trick!. Works for me and handles the problem good enough. Thanks for your msg.

Superb solution would be (don't ask me what, I think it doesn't exist) something that could customize the resource while destroying. omething like : "If [destroying] then parse all existing records, EXCLUDE my verification token, and save the rest." And after apply, discard the tfstate resource. I know, that's all logic beyond the design pattern of Terraform. But using null providers and that dirty hacks, maybe we could do some create-maintain-delete lifecycle for this scenarios: multi-account DNS maintenance. (SES, ACM, etc)

thehale commented 4 months ago

I ran into this limitation today too. In my case, I was trying to replace a TXT record with a consistent prefix (specifically a SPF record) so I was able to update the record in-place.

# Read the existing TXT record from DNS
# See the DNS provider: https://registry.terraform.io/providers/hashicorp/dns/latest/docs
data dns_txt_record_set example_txt {
  host = "example.com"
}

# Declare the desired SPF record
locals {
  spf_record = "v=spf1 ?all"
}

# Create/update the SPF record, preserving all other TXT records.
resource aws_route53_record spf_txt {
  allow_overwrite = true
  zone_id         = data.aws_route53_zone.example_com.id
  name            = "example.com"
  ttl             = 600
  type            = "TXT"

  # ⭐ HERE'S THE MAGIC BIT ⭐
  records = toset(concat([  # 💡 Deduplicate the TXT records
    for r in data.dns_txt_record_set.example_txt.records :
    startswith(r, "v=spf1") ? local.spf_record : r  # 💡(update) Replace existing SPF record(s) with the declared one
  ], [local.spf_record]))  # 💡 (create) Add the SPF record to the list
  # ⭐ END MAGIC ⭐

  lifecycle {
    prevent_destroy = true
  }
}

This approach ensures the desired SPF record appears once in the TXT record set, preserving all other existing TXT records. The for expression replaces all existing SPF records with the declared one. Then, the declared SPF record is concatenated into the list to handle the case where no SPF record yet exists. But since the concat duplicates the SPF record when it already exists, toset is required to remove that duplication.

Note that this approach fails when a TXT entry has more than 255 characters since that entry will span multiple records, resulting in only a partial replacement.

hashicorp / terraform-provider-aws