While creating tanzu-mission-control_tanzu_kubernetes_cluster, Terraform crashed after ~18 mins with the following error. Even if terraform crashed, the clusters were created in TMC.
tanzu-mission-control_tanzu_kubernetes_cluster.tap_view: Still creating... [18m30s elapsed]
tanzu-mission-control_tanzu_kubernetes_cluster.tap_run: Still creating... [18m30s elapsed]
tanzu-mission-control_tanzu_kubernetes_cluster.tap_build: Still creating... [18m30s elapsed]
tanzu-mission-control_tanzu_kubernetes_cluster.tap_iterate: Still creating... [18m30s elapsed]
╷
│ Error: Plugin did not respond
│
│ with tanzu-mission-control_tanzu_kubernetes_cluster.tap_build,
│ on clusters-tap-build.tf line 18, in resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_build":
│ 18: resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_build" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call.
│ The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│ with tanzu-mission-control_tanzu_kubernetes_cluster.tap_iterate,
│ on clusters-tap-iterate.tf line 18, in resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_iterate":
│ 18: resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_iterate" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call.
│ The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│ with tanzu-mission-control_tanzu_kubernetes_cluster.tap_run,
│ on clusters-tap-run.tf line 18, in resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_run":
│ 18: resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_run" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call.
│ The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│ with tanzu-mission-control_tanzu_kubernetes_cluster.tap_view,
│ on clusters-tap-view.tf line 18, in resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_view":
│ 18: resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_view" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call.
│ The plugin logs may contain more details.
╵
Releasing state lock. This may take a few moments...
Stack trace from the terraform-provider-tanzu-mission-control_v1.4.4 plugin:
fatal error: concurrent map writes
goroutine 52 [running]:
net/textproto.MIMEHeader.Set(...)
net/textproto/header.go:22
net/http.Header.Set(...)
net/http/header.go:40
github.com/vmware/terraform-provider-tanzu-mission-control/internal/authctx.refreshSMUserAuthCtx(0x140001b70e0)
github.com/vmware/terraform-provider-tanzu-mission-control/internal/authctx/selfmanaged.go:236 +0x11c
github.com/vmware/terraform-provider-tanzu-mission-control/internal/authctx.glob..func1(0x140001b70e0, 0x14000b2c618?, {0x1039fd580?, 0x14000b2c618?})
github.com/vmware/terraform-provider-tanzu-mission-control/internal/authctx/helper.go:117 +0x54
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster.waitClusterReady({0x103a134a8, 0x140001b7260}, 0x140001b70e0, 0x14000abdc80)
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster/helper.go:133 +0x32c
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster.readFullClusterResourceWait({0x103a134a8, 0x140001b7260}, 0x10331a63a?, 0xf?, {0x140007e97c0, 0x1, 0x0?}, 0x1)
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster/helper.go:68 +0x60
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster.readResourceWait({0x103a134e0, 0x140005c64b0}, 0x10484fd40?, 0x14000a18bc0?, {0x140007e97c0, 0x1, 0x4}, 0x63?)
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster/helper.go:45 +0x15c
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster.resourceTanzuKubernetesClusterRead({0x103a134e0, 0x140005c64b0}, 0x14000960e80, {0x1038fa8c0?, 0x1400059a180})
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster/resource_tanzu_kuberenetes_cluster.go:125 +0x2d8
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster.resourceTanzuKubernetesClusterCreate({0x103a134e0, 0x1400041ef30}, 0x0?, {0x1038fa8c0?, 0x1400059a180?})
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster/resource_tanzu_kuberenetes_cluster.go:103 +0x5d8
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0x103a134e0?, {0x103a134e0?, 0x1400041ef30?}, 0xd?, {0x1038fa8c0?, 0x1400059a180?})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/helper/schema/resource.go:702 +0x64
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0x1400077da40, {0x103a134e0, 0x1400041ef30}, 0x14000782b60, 0x14000693180, {0x1038fa8c0, 0x1400059a180})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/helper/schema/resource.go:837 +0x86c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0x14000780630, {0x103a13438?, 0x14000b60700?}, 0x14000180230)
github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/helper/schema/grpc_provider.go:1021 +0xb70
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0x1400047b0e0, {0x103a134e0?, 0x1400041e900?}, 0x140009060e0)
github.com/hashicorp/terraform-plugin-go@v0.9.0/tfprotov5/tf5server/server.go:812 +0x384
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0x103915180?, 0x1400047b0e0}, {0x103a134e0, 0x1400041e900}, 0x14000906070, 0x0)
github.com/hashicorp/terraform-plugin-go@v0.9.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:385 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0x140007681e0, {0x103a1a378, 0x14000502d00}, 0x140007dc360, 0x140006a7ef0, 0x1047daf00, 0x0)
google.golang.org/grpc@v1.56.3/server.go:1335 +0xc64
google.golang.org/grpc.(*Server).handleStream(0x140007681e0, {0x103a1a378, 0x14000502d00}, 0x140007dc360, 0x0)
google.golang.org/grpc@v1.56.3/server.go:1712 +0x82c
google.golang.org/grpc.(*Server).serveStreams.func1.1()
google.golang.org/grpc@v1.56.3/server.go:947 +0xb4
created by google.golang.org/grpc.(*Server).serveStreams.func1
google.golang.org/grpc@v1.56.3/server.go:958 +0x174
goroutine 1 [select, 18 minutes]:
github.com/hashicorp/go-plugin.Serve(0x140001b61e0)
github.com/hashicorp/go-plugin@v1.4.3/server.go:469 +0x10a8
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.Serve({0x10330fccd, 0x8}, 0x1400004c230, {0x0, 0x0, 0x0})
github.com/hashicorp/terraform-plugin-go@v0.9.0/tfprotov5/tf5server/server.go:311 +0x9e0
github.com/hashicorp/terraform-plugin-sdk/v2/plugin.tf5serverServe(0x140001b6180)
github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/plugin/serve.go:178 +0x4d4
github.com/hashicorp/terraform-plugin-sdk/v2/plugin.Serve(0x140001b6180)
github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/plugin/serve.go:118 +0x190
main.main()
github.com/vmware/terraform-provider-tanzu-mission-control/main.go:31 +0xf4
goroutine 9 [select, 18 minutes]:
github.com/hashicorp/go-plugin.(*gRPCBrokerServer).Recv(0x0?)
github.com/hashicorp/go-plugin@v1.4.3/grpc_broker.go:121 +0x58
github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0x14000180e10)
github.com/hashicorp/go-plugin@v1.4.3/grpc_broker.go:411 +0x40
created by github.com/hashicorp/go-plugin.(*GRPCServer).Init
github.com/hashicorp/go-plugin@v1.4.3/grpc_server.go:85 +0x424
goroutine 10 [IO wait, 18 minutes]:
internal/poll.runtime_pollWait(0x12c728350, 0x72)
runtime/netpoll.go:305 +0xa0
internal/poll.(*pollDesc).wait(0x140001b6660?, 0x14000225000?, 0x1)
internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140001b6660, {0x14000225000, 0x1000, 0x1000})
internal/poll/fd_unix.go:167 +0x1e0
os.(*File).read(...)
os/file_posix.go:31
os.(*File).Read(0x1400000e980, {0x14000225000?, 0x400?, 0x10371c7a0?})
os/file.go:119 +0x5c
bufio.(*Reader).Read(0x1400008bf30, {0x14000534000, 0x400, 0x0?})
bufio/bufio.go:237 +0x1e0
github.com/hashicorp/go-plugin.copyChan({0x103a1e178, 0x1400026a7e0}, 0x0?, {0x1039fcb60?, 0x1400000e980?})
github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:181 +0x154
created by github.com/hashicorp/go-plugin.newGRPCStdioServer
github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:37 +0x10c
goroutine 11 [IO wait, 18 minutes]:
internal/poll.runtime_pollWait(0x12c728170, 0x72)
runtime/netpoll.go:305 +0xa0
internal/poll.(*pollDesc).wait(0x140001b6720?, 0x14000536000?, 0x1)
internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140001b6720, {0x14000536000, 0x1000, 0x1000})
internal/poll/fd_unix.go:167 +0x1e0
os.(*File).read(...)
os/file_posix.go:31
os.(*File).Read(0x1400000e9a0, {0x14000536000?, 0x400?, 0x10371c7a0?})
os/file.go:119 +0x5c
bufio.(*Reader).Read(0x14000086730, {0x14000534400, 0x400, 0x0?})
bufio/bufio.go:237 +0x1e0
github.com/hashicorp/go-plugin.copyChan({0x103a1e178, 0x1400026a7e0}, 0x0?, {0x1039fcb60?, 0x1400000e9a0?})
github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:181 +0x154
created by github.com/hashicorp/go-plugin.newGRPCStdioServer
github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:38 +0x198
goroutine 37 [syscall, 18 minutes]:
Error: The terraform-provider-tanzu-mission-control_v1.4.4 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.
Describe the bug
While creating tanzu-mission-control_tanzu_kubernetes_cluster, Terraform crashed after ~18 mins with the following error. Even if terraform crashed, the clusters were created in TMC.
Reproduction steps
Unknown
I applied the terraform module several times in clean environments and I got this error only once source code is available here https://github.com/giovannibaratta/vmware-tanzu-training/tree/main/terraform/stages/50-tmc
Expected behavior
Terraform should not crash
Additional context
I am using TMC self managed v1.2 MacBook M1 (Terraform v1.7.4 on darwin_arm64)