search

hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io/
Other
42.31k stars 9.49k forks source link

Crash: `connection is shut down` #18689

Closed Miserlou closed 6 years ago

Miserlou commented 6 years ago

Terraform Version

~/Projects/data-refinery/infrastructure $ terraform --version
Terraform v0.11.7
+ provider.aws v1.11.0
+ provider.local v1.1.0
+ provider.template v1.0.0

Debug Output

Crash Output


panic: runtime error: invalid memory address or nil pointer dereference
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x293ec2a]
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4:
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: goroutine 1711 [running]:
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: github.com/terraform-providers/terraform-provider-aws/aws.flattenAwsLbTargetGroupResource(0xc420a32770, 0x3230a80, 0xc4201c0a00, 0xc42059c090, 0x0, 0xc420a75330)
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4:  /opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-aws/aws/resource_aws_lb_target_group.go:533 +0x5fa
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: github.com/terraform-providers/terraform-provider-aws/aws.resourceAwsLbTargetGroupRead(0xc420a32770, 0x3230a80, 0xc4201c0a00, 0x0, 0x0)
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4:  /opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-aws/aws/resource_aws_lb_target_group.go:273 +0x28b
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: github.com/terraform-providers/terraform-provider-aws/aws.resourceAwsLbTargetGroupUpdate(0xc420a32770, 0x3230a80, 0xc4201c0a00, 0x0, 0x0)
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4:  /opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-aws/aws/resource_aws_lb_target_group.go:375 +0x178
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: github.com/terraform-providers/terraform-provider-aws/aws.resourceAwsLbTargetGroupCreate(0xc420a32770, 0x3230a80, 0xc4201c0a00, 0xc420a32770, 0x0)
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4:  /opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-aws/aws/resource_aws_lb_target_group.go:251 +0x8f9
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(*Resource).Apply(0xc4202a9f80, 0xc420839db0, 0xc4205dd7a0, 0x3230a80, 0xc4201c0a00, 0x1, 0xc4205cbef0, 0xc420c82ad8)
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4:  /opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/resource.go:227 +0x364
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(*Provider).Apply(0xc420bc47e0, 0xc420839d60, 0xc420839db0, 0xc4205dd7a0, 0x61738b0, 0x0, 0x2e5)
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4:  /opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/provider.go:283 +0xa4
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/plugin.(*ResourceProviderServer).Apply(0xc42045b7a0, 0xc4205dd740, 0xc420480230, 0x0, 0x0)
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4:  /opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/plugin/resource_provider.go:527 +0x57
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: reflect.Value.call(0xc4201d8780, 0xc4203b2010, 0x13, 0x364b02f, 0x4, 0xc420c82f20, 0x3, 0x3, 0x0, 0x0, ...)
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4:  /usr/local/go/src/reflect/value.go:434 +0x905
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: reflect.Value.Call(0xc4201d8780, 0xc4203b2010, 0x13, 0xc4205a7f20, 0x3, 0x3, 0xc42098bc20, 0xc420722240, 0xc4203e2640)
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4:  /usr/local/go/src/reflect/value.go:302 +0xa4
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: net/rpc.(*service).call(0xc42097a140, 0xc420838000, 0xc4207f2060, 0xc42085ef80, 0xc4204585a0, 0x2d7cf40, 0xc4205dd740, 0x16, 0x2d7cf80, 0xc420480230, ...)
2018-08-15T16:06:29.376-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4:  /usr/local/go/src/net/rpc/server.go:381 +0x142
2018-08-15T16:06:29.377-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4: created by net/rpc.(*Server).ServeCodec
2018-08-15T16:06:29.377-0400 [DEBUG] plugin.terraform-provider-aws_v1.11.0_x4:  /usr/local/go/src/net/rpc/server.go:475 +0x36b
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyProvisioners
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalIf
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyProvisioners
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalIf
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyProvisioners
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalIf
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteDiff
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyPost
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyProvisioners
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalIf
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteDiff
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyPost
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyProvisioners
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_lb_target_group.api-http: unexpected EOF
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_lb_target_group.api-http: unexpected EOF
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_acm_certificate_validation.ssl-cert: unexpected EOF
2018/08/15 16:06:29 [TRACE] [walkApply] Exiting eval tree: aws_lb_target_group.api-http
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalIf
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_acm_certificate_validation.ssl-cert: unexpected EOF
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] [walkApply] Exiting eval tree: aws_acm_certificate_validation.ssl-cert
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteDiff
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyPost
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_subnet.data_refinery_1b: unexpected EOF
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_cloudfront_distribution.static-distribution"
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyProvisioners
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalIf
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteDiff
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyPost
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyProvisioners
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalIf
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteDiff
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_security_group.data_refinery_db: unexpected EOF
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_security_group.data_refinery_db: unexpected EOF
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] [walkApply] Exiting eval tree: aws_security_group.data_refinery_db
2018-08-15T16:06:29.380-0400 [DEBUG] plugin: plugin process exited: path=/Users/rjones/Projects/data-refinery/infrastructure/.terraform/plugins/darwin_amd64/terraform-provider-aws_v1.11.0_x4
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyProvisioners
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalIf
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteDiff
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyPost
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_ci_postgres"
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_db_outbound"
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteDiff
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyPost
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalSequence
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalInstanceInfo
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalReadDiff
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_subnet.data_refinery_1b: unexpected EOF
2018/08/15 16:06:29 [TRACE] [walkApply] Exiting eval tree: aws_subnet.data_refinery_1b
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteDiff
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyPost
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_internet_gateway.data_refinery: unexpected EOF
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyProvisioners
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalIf
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteDiff
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyPost
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyPost
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_lb_target_group.api-https: unexpected EOF
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_security_group.data_refinery_worker: unexpected EOF
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_security_group.data_refinery_worker: unexpected EOF
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_lb_target_group.api-https: unexpected EOF
2018/08/15 16:06:29 [TRACE] [walkApply] Exiting eval tree: aws_lb_target_group.api-https
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyProvisioners
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalIf
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_security_group.data_refinery_api: unexpected EOF
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteState
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_security_group.data_refinery_api: unexpected EOF
2018/08/15 16:06:29 [TRACE] [walkApply] Exiting eval tree: aws_security_group.data_refinery_api
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalWriteDiff
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalApplyPost
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_api_http"
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalIf
2018/08/15 16:06:29 [TRACE] root: eval: terraform.EvalNoop
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_api_https"
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalIf
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_subnet.data_refinery_1a: unexpected EOF
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_subnet.data_refinery_1a: unexpected EOF
2018/08/15 16:06:29 [TRACE] [walkApply] Exiting eval tree: aws_subnet.data_refinery_1a
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_api_ssh"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_lb.data_refinery_api_load_balancer"
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalInterpolate
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_lb_listener.api-http"
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_internet_gateway.data_refinery: unexpected EOF
2018/08/15 16:06:29 [TRACE] [walkApply] Exiting eval tree: aws_internet_gateway.data_refinery
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_lb_listener.api-https"
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_security_group.data_refinery_foreman: unexpected EOF
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_security_group.data_refinery_foreman: unexpected EOF
2018/08/15 16:06:29 [TRACE] [walkApply] Exiting eval tree: aws_security_group.data_refinery_foreman
2018/08/15 16:06:29 [TRACE] [walkApply] Exiting eval tree: aws_security_group.data_refinery_worker
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_db_api_icmp"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_foreman_ssh"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_db_api_tcp"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_nomad_from_foreman"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_api_outbound"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_foreman_outbound"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_db_subnet_group.data_refinery"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_route_table.data_refinery"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_worker_nfs"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_db_foreman_tcp"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_db_workers_tcp"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_worker_custom"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_statsd"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_worker_nomad_rpc"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_worker_foreman"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_efs_mount_target.data_refinery_efs_1a"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_worker_nomad_serf_tcp"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_worker_ssh"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_worker_nomad"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_instance.nomad_server_1"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "output.nomad_server_1_ip"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_worker_outbound"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_route_table_association.data_refinery_1a"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_ci_nomad"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_route_table_association.data_refinery_1b"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "data.template_file.nomad_server_config"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_worker_nomad_serf_udp"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "data.template_file.nomad_server_script_smusher"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_efs_mount_target.data_refinery_efs_1b"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_api_nomad"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_instance.nomad_server_2"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "data.template_file.nomad_client_config"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_instance.nomad_server_3"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_security_group_rule.data_refinery_db_workers_icmp"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "output.nomad_server_3_ip"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "output.nomad_server_2_ip"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_db_instance.postgres_db"
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalInterpolate
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "data.template_file.pg_bouncer_script_smusher"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "output.environment_variables"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_instance.pg_bouncer"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "data.template_file.nomad_client_script_smusher"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_launch_configuration.auto_client_configuration"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "data.template_file.api_server_script_smusher"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "data.template_file.foreman_server_script_smusher"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_instance.api_server_1"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_autoscaling_group.clients"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "output.api_server_1_ip"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_autoscaling_policy.clients_scale_up"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_autoscaling_policy.clients_scale_down"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_lb_target_group_attachment.api-https"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_cloudwatch_metric_alarm.nomad_queue_length_alarm_down"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_cloudwatch_metric_alarm.nomad_queue_length_alarm_up"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_instance.foreman_server_1"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "aws_lb_target_group_attachment.api-http"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "provider.template (close)"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "output.foreman_server_1_ip"
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalGetProvider
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalReadState
2018/08/15 16:06:29 [TRACE] root: eval: *terraform.EvalValidateResource
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalValidateResource, err: Warnings: []. Errors: [connection is shut down]
2018/08/15 16:06:29 [ERROR] root: eval: *terraform.EvalSequence, err: Warnings: []. Errors: [connection is shut down]
2018/08/15 16:06:29 [TRACE] [walkApply] Exiting eval tree: aws_iam_user_policy.data-refinery-deployer
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "meta.count-boundary (count boundary fixup)"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "provider.aws (close)"
2018/08/15 16:06:29 [TRACE] dag/walk: upstream errored, not walking "root"
2018/08/15 16:06:29 [TRACE] Preserving existing state lineage "da0e8950-f7da-4041-a37e-ee62d56ff10a"
2018/08/15 16:06:29 [TRACE] Preserving existing state lineage "da0e8950-f7da-4041-a37e-ee62d56ff10a"
2018/08/15 16:06:29 [TRACE] Preserving existing state lineage "da0e8950-f7da-4041-a37e-ee62d56ff10a"
2018/08/15 16:06:29 [DEBUG] Uploading remote state to S3: {
  Body: buffer(0xc420807140),
  Bucket: "refinebio-tfstate-deploy-dev",
  ContentLength: 82260,
  ContentType: "application/json",
  Key: "terraform-rjpgb2.tfstate"
}
2018/08/15 16:06:29 [DEBUG] plugin: waiting for all plugin processes to complete...
2018-08-15T16:06:29.595-0400 [WARN ] plugin: error closing client during Kill: err="connection is shut down"
2018-08-15T16:06:29.598-0400 [DEBUG] plugin: plugin process exited: path=/Users/rjones/Projects/data-refinery/infrastructure/.terraform/plugins/darwin_amd64/terraform-provider-local_v1.1.0_x4
2018-08-15T16:06:29.598-0400 [DEBUG] plugin: plugin process exited: path=/Users/rjones/Projects/data-refinery/infrastructure/.terraform/plugins/darwin_amd64/terraform-provider-template_v1.0.0_x4

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
A crash log has been placed at "crash.log" relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version. That
information is available on the first line of crash.log. You can also
get it by running 'terraform --version' on the command line.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Expected Behavior

Don't crash.

Actual Behavior

Crashes every single time. Have to manually remove 99 items via the AWS console. Fuck this garbage stack. Fuck my life.

Steps to Reproduce

terraform apply

apparentlymart commented 6 years ago

Hi @Miserlou,

First, I want to apologize for this crash and the manual work it created for you. I realize that when Terraform crashes it always harms your confidence in the product, especially when it leaves you in a bad state with resources that are not tracked. Responding to crashes with a prompt fix is a high priority for us because of the disruption they can cause.

I worked with the AWS provider maintainer team here and we found that this issue was reported in terraform-providers/terraform-provider-aws#3949 and subsequently fixed in AWS Provider version 1.13.0. Of course that doesn't change the fact that this crash affected you with the version you are currently running, and caused you to have to manually delete a lot of items that Terraform had already created.

We do the best we can to verify both Terraform Core and AWS provider releases, but due to the complexity of interacting with constantly-changing remote systems, unfortunately sometimes bugs like this creep in only after a release. In this particular case, the original behavior of the underlying ELB API masked a bug in the AWS provider that was discovered only after that behavior changed on the AWS side, exposing a lack of robustness in the provider's handling of the API responses.

As a result of this issue, the provider maintainer team has increased their vigilance for this class of bug when reviewing changes to the provider, to reduce the likelihood that future API changes will expose response parsing bugs. The team is also working through existing code to identify and fix previous bugs of this type. We have automated regression tests that run nightly in an attempt to catch regressions, though of course we can't necessarily anticipate how the API will change and so a combination of those tests and care during review are how we've responded to this problem.

Sorry again for the inconvenience caused by this crash. I believe upgrading to AWS Provider v1.13.0 or later will prevent thus problem from occurring for you again. Please let me know if you run into problems with that upgrade, and I'll ask the AWS provider team for their input.

Miserlou commented 6 years ago

@apparentlymart - thank you very much for your swift and thorough reply! Upgrading the provider version seems to have solved the issue.

 provider "aws" {
    version = "1.31.0"
    region = "${var.region}"
 }
ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.