search

hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io/
Other
42.68k stars 9.55k forks source link

Terrafrom Crash #25742

Closed ed-hammond closed 4 years ago

ed-hammond commented 4 years ago

Terraform Version

This works:

Terraform v0.12.29
+ provider.google v3.23.0
+ provider.null v2.1.2

This causes the crash:

Terraform v0.12.29
+ provider.google v3.32.0              << note latest release (downloaded to clean directory)
+ provider.null v2.1.2

Terraform Configuration Files

Debug Output

Error: rpc error: code = Unavailable desc = transport is closing

panic: nil context 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: goroutine 16 [running]: 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: net/http.(Request).WithContext(...) 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/goenv/versions/1.14.5/src/net/http/request.go:358 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: golang.org/x/net/context/ctxhttp.Do(0x0, 0x0, 0x3bbe6a0, 0xc000136700, 0x788c769d3bebe7f2, 0x10, 0x219f400) 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/golang.org/x/net@v0.0.0-20200324143707-d3edc9973b7e/context/ctxhttp/ctxhttp.go:27 +0x32b 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: golang.org/x/oauth2/internal.doTokenRoundTrip(0x0, 0x0, 0xc000136700, 0x26, 0xc000035ae0, 0x18) 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/golang.org/x/oauth2@v0.0.0-20200107190931-bf48bf16ab8d/internal/token.go:232 +0x82 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: golang.org/x/oauth2/internal.RetrieveToken(0x0, 0x0, 0xc00018aff0, 0x26, 0xc000035ae0, 0x18, 0x27c560d, 0x23, 0xc0001b32a0, 0x1, ...) 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/golang.org/x/oauth2@v0.0.0-20200107190931-bf48bf16ab8d/internal/token.go:202 +0xe6 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: golang.org/x/oauth2.retrieveToken(0x0, 0x0, 0xc0006fe0e0, 0xc0001b32a0, 0xc0001b3370, 0x1a, 0x1) 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/golang.org/x/oauth2@v0.0.0-20200107190931-bf48bf16ab8d/token.go:157 +0x99 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: golang.org/x/oauth2.(tokenRefresher).Token(0xc0006f7170, 0x4adbf4, 0x27864c1, 0x6) 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/golang.org/x/oauth2@v0.0.0-20200107190931-bf48bf16ab8d/oauth2.go:270 +0x1e5 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: golang.org/x/oauth2.(reuseTokenSource).Token(0xc0007cb380, 0x0, 0x0, 0x0) 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/golang.org/x/oauth2@v0.0.0-20200107190931-bf48bf16ab8d/oauth2.go:304 +0xaf 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: golang.org/x/oauth2.(Transport).RoundTrip(0xc0007cb3c0, 0xc000136600, 0x0, 0x0, 0x0) 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/golang.org/x/oauth2@v0.0.0-20200107190931-bf48bf16ab8d/transport.go:45 +0xa5 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: github.com/hashicorp/terraform-plugin-sdk/helper/logging.(transport).RoundTrip(0xc0007cb3e0, 0xc000136600, 0xc0001b3610, 0x1, 0x1) 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.11.0/helper/logging/transport.go:27 +0x16b 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: github.com/terraform-providers/terraform-provider-google/google.(retryTransport).RoundTrip(0xc0006f7230, 0xc000136400, 0x0, 0x0, 0x0) 2020-08-04T10:41:42.035-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-google/google/retry_transport.go:125 +0x2cc 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: net/http.send(0xc000136100, 0x2c36ea0, 0xc0006f7230, 0xbfc27ad101e35854, 0x7138733d1, 0x3bbe9e0, 0xc000a20e18, 0xbfc27ad101e35854, 0x1, 0x0) 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/goenv/versions/1.14.5/src/net/http/client.go:252 +0x445 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: net/http.(Client).send(0xc0006f7200, 0xc000136100, 0xbfc27ad101e35854, 0x7138733d1, 0x3bbe9e0, 0xc000a20e18, 0x0, 0x1, 0x2030001) 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/goenv/versions/1.14.5/src/net/http/client.go:176 +0x101 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: net/http.(Client).do(0xc0006f7200, 0xc000136100, 0x0, 0x0, 0x0) 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/goenv/versions/1.14.5/src/net/http/client.go:699 +0x451 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: net/http.(Client).Do(...) 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/goenv/versions/1.14.5/src/net/http/client.go:567 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: google.golang.org/api/internal/gensupport.SendRequest(0x0, 0x0, 0xc0006f7200, 0xc000136100, 0xc000086540, 0x59, 0x2c36220) 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/google.golang.org/api@v0.26.0/internal/gensupport/send.go:42 +0x244 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: google.golang.org/api/storage/v1.(BucketsInsertCall).doRequest(0xc0001b3d50, 0x2784c0d, 0x4, 0x0, 0xb88413a501d9cb01, 0xc000342160) 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/google.golang.org/api@v0.26.0/storage/v1/storage-gen.go:4076 +0x77d 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: google.golang.org/api/storage/v1.(BucketsInsertCall).Do(0xc0009a3d50, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/google.golang.org/api@v0.26.0/storage/v1/storage-gen.go:4088 +0xa8 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: github.com/terraform-providers/terraform-provider-google/google.resourceStorageBucketCreate.func1(0xc0002079c8, 0xc0009a3ea4) 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-google/google/resource_storage_bucket.go:429 +0x185 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: github.com/terraform-providers/terraform-provider-google/google.retryTimeDuration.func1(0x495997) 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-google/google/retry_utils.go:21 +0x39 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: github.com/hashicorp/terraform-plugin-sdk/helper/resource.Retry.func1(0x0, 0x0, 0x0, 0x0, 0x0, 0x0) 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.11.0/helper/resource/wait.go:22 +0x65 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: github.com/hashicorp/terraform-plugin-sdk/helper/resource.(StateChangeConf).WaitForState.func1(0xc000207860, 0xc0006fed20, 0xc00069a840, 0xc000086120, 0xc0009df380, 0xc0009df378) 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.11.0/helper/resource/state.go:103 +0x29f 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: created by github.com/hashicorp/terraform-plugin-sdk/helper/resource.(StateChangeConf).WaitForState 2020-08-04T10:41:42.036-0400 [DEBUG] plugin.terraform-provider-google_v3.32.0_x5.exe: /opt/teamcity-agent/work/5d79fe75d4460a2f/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.11.0/helper/resource/state.go:80 +0x1c6 2020/08/04 10:41:42 [DEBUG] google_storage_bucket.thisBucket: apply errored, but we're indicating that via the Error pointer rather than returning it: rpc error: code = Unavailable desc = transport is closing 2020/08/04 10:41:42 [TRACE] : eval: terraform.EvalMaybeTainted 2020/08/04 10:41:42 [TRACE] EvalMaybeTainted: google_storage_bucket.thisBucket encountered an error during creation, so it is now marked as tainted 2020/08/04 10:41:42 [TRACE] : eval: terraform.EvalWriteState 2020/08/04 10:41:42 [TRACE] EvalWriteState: removing state object for google_storage_bucket.thisBucket 2020/08/04 10:41:42 [TRACE] : eval: terraform.EvalApplyProvisioners 2020/08/04 10:41:42 [TRACE] EvalApplyProvisioners: google_storage_bucket.thisBucket has no state, so skipping provisioners 2020/08/04 10:41:42 [TRACE] : eval: terraform.EvalMaybeTainted 2020/08/04 10:41:42 [TRACE] EvalMaybeTainted: google_storage_bucket.thisBucket encountered an error during creation, so it is now marked as tainted 2020/08/04 10:41:42 [TRACE] : eval: terraform.EvalWriteState 2020/08/04 10:41:42 [TRACE] EvalWriteState: removing state object for google_storage_bucket.thisBucket 2020/08/04 10:41:42 [TRACE] : eval: terraform.EvalIf 2020/08/04 10:41:42 [TRACE] : eval: terraform.EvalIf 2020/08/04 10:41:42 [TRACE] : eval: terraform.EvalWriteDiff 2020/08/04 10:41:42 [TRACE] : eval: terraform.EvalApplyPost 2020/08/04 10:41:42 [ERROR] : eval: terraform.EvalApplyPost, err: rpc error: code = Unavailable desc = transport is closing 2020/08/04 10:41:42 [ERROR] : eval: terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing 2020/08/04 10:41:42 [TRACE] [walkApply] Exiting eval tree: google_storage_bucket.thisBucket 2020/08/04 10:41:42 [TRACE] vertex "google_storage_bucket.thisBucket": visit complete 2020/08/04 10:41:42 [TRACE] dag/walk: upstream of "output.bucket_name" errored, so skipping 2020/08/04 10:41:42 [TRACE] dag/walk: upstream of "google_service_account.thisSA" errored, so skipping 2020/08/04 10:41:42 [TRACE] dag/walk: upstream of "output.service_account_name" errored, so skipping 2020/08/04 10:41:42 [TRACE] dag/walk: upstream of "google_storage_bucket_iam_member.thisBucket-thisSA" errored, so skipping 2020/08/04 10:41:42 [TRACE] dag/walk: upstream of "provider.google (close)" errored, so skipping 2020/08/04 10:41:42 [TRACE] dag/walk: upstream of "null_resource.createJsonKey" errored, so skipping 2020/08/04 10:41:42 [TRACE] dag/walk: upstream of "meta.count-boundary (EachMode fixup)" errored, so skipping 2020/08/04 10:41:42 [TRACE] dag/walk: upstream of "provider.null (close)" errored, so skipping 2020/08/04 10:41:42 [TRACE] dag/walk: upstream of "provisioner.local-exec (close)" errored, so skipping 2020/08/04 10:41:42 [TRACE] dag/walk: upstream of "root" errored, so skipping 2020/08/04 10:41:42 [TRACE] statemgr.Filesystem: reading latest snapshot from terraform.tfstate 2020/08/04 10:41:42 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay 2020/08/04 10:41:42 [TRACE] statemgr.Filesystem: read nil snapshot 2020/08/04 10:41:42 [TRACE] statemgr.Filesystem: no original state snapshot to back up 2020/08/04 10:41:42 [TRACE] statemgr.Filesystem: state has changed since last snapshot, so incrementing serial to 1 2020/08/04 10:41:42 [TRACE] statemgr.Filesystem: writing snapshot at terraform.tfstate 2020-08-04T10:41:42.054-0400 [DEBUG] plugin: plugin process exited: path=C:\Users\XXXXXXXXXXXXX\AppData\Local\Temp\tmpz.terraform\plugins\windows_amd64\terraform-provider-google_v3.32.0_x5.exe pid=10856 error="exit status 2" 2020/08/04 10:41:42 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info 2020/08/04 10:41:42 [TRACE] statemgr.Filesystem: unlocked by closing terraform.tfstate 2020-08-04T10:41:42.058-0400 [DEBUG] plugin: plugin exited 2020-08-04T10:41:42.059-0400 [WARN] plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing" 2020-08-04T10:41:42.074-0400 [DEBUG] plugin: plugin process exited: path=C:\Users\XXXXXXXXXXXXX\AppData\Local\Temp\tmpz.terraform\plugins\windows_amd64\terraform-provider-null_v2.1.2_x4.exe pid=18076 2020-08-04T10:41:42.074-0400 [DEBUG] plugin: plugin exited 2020-08-04T10:41:42.097-0400 [DEBUG] plugin: plugin process exited: path=C:\Users\XXXXXXXXXXXXXXXXX\bin\terraform.exe pid=21716 2020-08-04T10:41:42.097-0400 [DEBUG] plugin: plugin exited

Crash Output

https://gist.github.com/cah-ed-hammond/dde36b23ac382ddd2db6ba57d568d2e3

Expected Behavior

Create resources as I was able to three weeks ago.
It works with Google plugin v3.23.0 while v3.32.0 crashes.

Actual Behavior

See crash Gist

Steps to Reproduce

Started with clean directory, downloaded latest Terraform exe, downloaded latest Google plugin. terraform init and terraform plan work fine. terraform apply crashes whey "yes" is entered to proceed.

Additional Context

References

ed-hammond commented 4 years ago

Works with Google plugin v3.29.0 but breaks when you hit 30+. v3.33 just came out and it too has the same issue.

ghost commented 4 years ago

This issue has been automatically migrated to terraform-providers/terraform-provider-google#6940 because it looks like an issue with that provider. If you believe this is not an issue with the provider, please reply to terraform-providers/terraform-provider-google#6940.

ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.