Creating Route53 Record Hangs Indefinitely

tinyzimmer commented 6 years ago

I have an issue on the latest version of Terraform (and this has been working fine for about 2 weeks), where it will hang indefinitely while trying to create a route53 record. I can see in the console the record gets created and resolves correctly, however, terraform just hangs out until I interrupt. Below debug is just one loop, however, I let this hang for as long as 10 minutes.

Terraform Version

Terraform v0.11.7

provider.aws: version =  1.27
                 AND
provider.aws: version =  1.28

Terraform Configuration Files

Just your standard (I think) RDS setup, I redacted a few parts of the database config. I am adding a CNAME Route53 zone that remains constant for configuration in artifacts at the moment.

resource "aws_db_subnet_group" "db_subnets" {
  name       = "subnet-group"
  subnet_ids = ["${var.subnet_ids}"]
}

resource "aws_db_instance" "db" {
  allocated_storage    = 10
  storage_type         = "gp2"
  engine               = "mariadb"
  engine_version       = "10.2.15"
  instance_class       = "db.t2.micro"
  name                 = "mydb"
  username             = "supersecretusername"
  password             = "supersecretpassword"
  parameter_group_name = "default.mariadb"
  db_subnet_group_name = "${aws_db_subnet_group.db_subnets.name}"
}

resource "aws_route53_record" "database" {
  zone_id = "${var.route53_zone}"
  name = "db.${var.environment}.example.net"
  type = "CNAME"
  ttl = "300"
  records = ["${aws_db_instance.db.address}"]
}

Debug Output

module.database.aws_route53_record.database: Still creating... (50s elapsed)

2018/07/18 18:38:58 [TRACE] dag/walk: vertex "root", waiting for: "provider.aws (close)" 2018/07/18 18:38:58 [TRACE] dag/walk: vertex "meta.count-boundary (count boundary fixup)", waiting for: "module.database.aws_route53_record.database" 2018/07/18 18:38:58 [TRACE] dag/walk: vertex "provider.aws (close)", waiting for: "module.database.aws_route53_record.database"

2018-07-18T18:39:01.494-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: 2018/07/18 18:39:01 [DEBUG] [aws-sdk-go] DEBUG: Request route53/GetChange Details: 2018-07-18T18:39:01.494-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: ---[ REQUEST POST-SIGN ]----------------------------- 2018-07-18T18:39:01.494-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: GET /2013-04-01/change/[secret?] HTTP/1.1 2018-07-18T18:39:01.494-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: Host: route53.amazonaws.com 2018-07-18T18:39:01.494-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: User-Agent: aws-sdk-go/1.14.26 (go1.9.2; linux; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.11.8-dev 2018-07-18T18:39:01.494-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: Authorization: AWS4-HMAC-SHA256 Credential=redacted/us-east-1/route53/aws4_request, SignedHeaders=host;x-amz-date, Signature=redacted 2018-07-18T18:39:01.494-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: X-Amz-Date: 20180719T013901Z 2018-07-18T18:39:01.494-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: Accept-Encoding: gzip 2018-07-18T18:39:01.494-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: 2018-07-18T18:39:01.495-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: 2018-07-18T18:39:01.495-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: -----------------------------------------------------

2018-07-18T18:39:01.829-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: 2018/07/18 18:39:01 [DEBUG] [aws-sdk-go] DEBUG: Response route53/GetChange Details: 2018-07-18T18:39:01.829-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: ---[ RESPONSE ]-------------------------------------- 2018-07-18T18:39:01.830-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: HTTP/1.1 200 OK 2018-07-18T18:39:01.830-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: Connection: close 2018-07-18T18:39:01.830-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: Content-Length: 285 2018-07-18T18:39:01.830-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: Content-Type: text/xml 2018-07-18T18:39:01.830-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: Date: Thu, 19 Jul 2018 01:39:00 GMT 2018-07-18T18:39:01.830-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: X-Amzn-Requestid: 836fb72b-8af4-11e8-a81e-572a2010db76 2018-07-18T18:39:01.830-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: 2018-07-18T18:39:01.830-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: 2018-07-18T18:39:01.830-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: ----------------------------------------------------- 2018-07-18T18:39:01.830-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: 2018/07/18 18:39:01 [DEBUG] [aws-sdk-go] <?xml version="1.0"?> 2018-07-18T18:39:01.830-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: /change/redactedPENDING2018-07-19T01:38:05.321ZManaged by Terraform 2018-07-18T18:39:01.831-0700 [DEBUG] plugin.terraform-provider-aws_v1.28.0_x4: 2018/07/18 18:39:01 [TRACE] Waiting 10s before next try

2018/07/18 18:39:03 [TRACE] dag/walk: vertex "root", waiting for: "provider.aws (close)" 2018/07/18 18:39:03 [TRACE] dag/walk: vertex "meta.count-boundary (count boundary fixup)", waiting for: "module.database.aws_route53_record.database" 2018/07/18 18:39:03 [TRACE] dag/walk: vertex "provider.aws (close)", waiting for: "module.database.aws_route53_record.database"

module.database.aws_route53_record.database: Still creating... (1m0s elapsed)

Expected Behavior

Everything is as it should be, let's continue on...

Actual Behavior

No...it's mad about something...

Steps to Reproduce

Hopefully, the code above (with needed variables) can reproduce. If it doesn't well damn...I've tried everything.

Additional Context

This is being run from a Jenkins job. We had some DNS issues earlier internally, but they got resolved, and it is clear from the debug output that it is getting a response from Amazon.

ghost commented 6 years ago

This issue has been automatically migrated to terraform-providers/terraform-provider-aws#5245 because it looks like an issue with that provider. If you believe this is not an issue with the provider, please reply to terraform-providers/terraform-provider-aws#5245.

ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform