search

vmware / terraform-provider-tanzu-mission-control

Terraform Provider for VMware Tanzu Mission Control
https://registry.terraform.io/providers/vmware/tanzu-mission-control/latest/docs
Mozilla Public License 2.0
37 stars 33 forks source link

fatal error: concurrent map writes #397

Open giovannibaratta opened 7 months ago

giovannibaratta commented 7 months ago

Describe the bug

While creating tanzu-mission-control_tanzu_kubernetes_cluster, Terraform crashed after ~18 mins with the following error. Even if terraform crashed, the clusters were created in TMC.

tanzu-mission-control_tanzu_kubernetes_cluster.tap_view: Still creating... [18m30s elapsed]
tanzu-mission-control_tanzu_kubernetes_cluster.tap_run: Still creating... [18m30s elapsed]
tanzu-mission-control_tanzu_kubernetes_cluster.tap_build: Still creating... [18m30s elapsed]
tanzu-mission-control_tanzu_kubernetes_cluster.tap_iterate: Still creating... [18m30s elapsed]
╷
│ Error: Plugin did not respond
│ 
│   with tanzu-mission-control_tanzu_kubernetes_cluster.tap_build,
│   on clusters-tap-build.tf line 18, in resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_build":
│   18: resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_build" {
│ 
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call.
│ The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│ 
│   with tanzu-mission-control_tanzu_kubernetes_cluster.tap_iterate,
│   on clusters-tap-iterate.tf line 18, in resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_iterate":
│   18: resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_iterate" {
│ 
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call.
│ The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│ 
│   with tanzu-mission-control_tanzu_kubernetes_cluster.tap_run,
│   on clusters-tap-run.tf line 18, in resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_run":
│   18: resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_run" {
│ 
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call.
│ The plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│ 
│   with tanzu-mission-control_tanzu_kubernetes_cluster.tap_view,
│   on clusters-tap-view.tf line 18, in resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_view":
│   18: resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tap_view" {
│ 
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ApplyResourceChange call.
│ The plugin logs may contain more details.
╵
Releasing state lock. This may take a few moments...

Stack trace from the terraform-provider-tanzu-mission-control_v1.4.4 plugin:

fatal error: concurrent map writes

goroutine 52 [running]:
net/textproto.MIMEHeader.Set(...)
        net/textproto/header.go:22
net/http.Header.Set(...)
        net/http/header.go:40
github.com/vmware/terraform-provider-tanzu-mission-control/internal/authctx.refreshSMUserAuthCtx(0x140001b70e0)
        github.com/vmware/terraform-provider-tanzu-mission-control/internal/authctx/selfmanaged.go:236 +0x11c
github.com/vmware/terraform-provider-tanzu-mission-control/internal/authctx.glob..func1(0x140001b70e0, 0x14000b2c618?, {0x1039fd580?, 0x14000b2c618?})
        github.com/vmware/terraform-provider-tanzu-mission-control/internal/authctx/helper.go:117 +0x54
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster.waitClusterReady({0x103a134a8, 0x140001b7260}, 0x140001b70e0, 0x14000abdc80)
        github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster/helper.go:133 +0x32c
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster.readFullClusterResourceWait({0x103a134a8, 0x140001b7260}, 0x10331a63a?, 0xf?, {0x140007e97c0, 0x1, 0x0?}, 0x1)
        github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster/helper.go:68 +0x60
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster.readResourceWait({0x103a134e0, 0x140005c64b0}, 0x10484fd40?, 0x14000a18bc0?, {0x140007e97c0, 0x1, 0x4}, 0x63?)
        github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster/helper.go:45 +0x15c
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster.resourceTanzuKubernetesClusterRead({0x103a134e0, 0x140005c64b0}, 0x14000960e80, {0x1038fa8c0?, 0x1400059a180})
        github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster/resource_tanzu_kuberenetes_cluster.go:125 +0x2d8
github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster.resourceTanzuKubernetesClusterCreate({0x103a134e0, 0x1400041ef30}, 0x0?, {0x1038fa8c0?, 0x1400059a180?})
        github.com/vmware/terraform-provider-tanzu-mission-control/internal/resources/tanzukubernetescluster/resource_tanzu_kuberenetes_cluster.go:103 +0x5d8
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).create(0x103a134e0?, {0x103a134e0?, 0x1400041ef30?}, 0xd?, {0x1038fa8c0?, 0x1400059a180?})
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/helper/schema/resource.go:702 +0x64
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0x1400077da40, {0x103a134e0, 0x1400041ef30}, 0x14000782b60, 0x14000693180, {0x1038fa8c0, 0x1400059a180})
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/helper/schema/resource.go:837 +0x86c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0x14000780630, {0x103a13438?, 0x14000b60700?}, 0x14000180230)
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/helper/schema/grpc_provider.go:1021 +0xb70
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ApplyResourceChange(0x1400047b0e0, {0x103a134e0?, 0x1400041e900?}, 0x140009060e0)
        github.com/hashicorp/terraform-plugin-go@v0.9.0/tfprotov5/tf5server/server.go:812 +0x384
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0x103915180?, 0x1400047b0e0}, {0x103a134e0, 0x1400041e900}, 0x14000906070, 0x0)
        github.com/hashicorp/terraform-plugin-go@v0.9.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:385 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0x140007681e0, {0x103a1a378, 0x14000502d00}, 0x140007dc360, 0x140006a7ef0, 0x1047daf00, 0x0)
        google.golang.org/grpc@v1.56.3/server.go:1335 +0xc64
google.golang.org/grpc.(*Server).handleStream(0x140007681e0, {0x103a1a378, 0x14000502d00}, 0x140007dc360, 0x0)
        google.golang.org/grpc@v1.56.3/server.go:1712 +0x82c
google.golang.org/grpc.(*Server).serveStreams.func1.1()
        google.golang.org/grpc@v1.56.3/server.go:947 +0xb4
created by google.golang.org/grpc.(*Server).serveStreams.func1
        google.golang.org/grpc@v1.56.3/server.go:958 +0x174

goroutine 1 [select, 18 minutes]:
github.com/hashicorp/go-plugin.Serve(0x140001b61e0)
        github.com/hashicorp/go-plugin@v1.4.3/server.go:469 +0x10a8
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.Serve({0x10330fccd, 0x8}, 0x1400004c230, {0x0, 0x0, 0x0})
        github.com/hashicorp/terraform-plugin-go@v0.9.0/tfprotov5/tf5server/server.go:311 +0x9e0
github.com/hashicorp/terraform-plugin-sdk/v2/plugin.tf5serverServe(0x140001b6180)
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/plugin/serve.go:178 +0x4d4
github.com/hashicorp/terraform-plugin-sdk/v2/plugin.Serve(0x140001b6180)
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/plugin/serve.go:118 +0x190
main.main()
        github.com/vmware/terraform-provider-tanzu-mission-control/main.go:31 +0xf4

goroutine 9 [select, 18 minutes]:
github.com/hashicorp/go-plugin.(*gRPCBrokerServer).Recv(0x0?)
        github.com/hashicorp/go-plugin@v1.4.3/grpc_broker.go:121 +0x58
github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0x14000180e10)
        github.com/hashicorp/go-plugin@v1.4.3/grpc_broker.go:411 +0x40
created by github.com/hashicorp/go-plugin.(*GRPCServer).Init
        github.com/hashicorp/go-plugin@v1.4.3/grpc_server.go:85 +0x424

goroutine 10 [IO wait, 18 minutes]:
internal/poll.runtime_pollWait(0x12c728350, 0x72)
        runtime/netpoll.go:305 +0xa0
internal/poll.(*pollDesc).wait(0x140001b6660?, 0x14000225000?, 0x1)
        internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140001b6660, {0x14000225000, 0x1000, 0x1000})
        internal/poll/fd_unix.go:167 +0x1e0
os.(*File).read(...)
        os/file_posix.go:31
os.(*File).Read(0x1400000e980, {0x14000225000?, 0x400?, 0x10371c7a0?})
        os/file.go:119 +0x5c
bufio.(*Reader).Read(0x1400008bf30, {0x14000534000, 0x400, 0x0?})
        bufio/bufio.go:237 +0x1e0
github.com/hashicorp/go-plugin.copyChan({0x103a1e178, 0x1400026a7e0}, 0x0?, {0x1039fcb60?, 0x1400000e980?})
        github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:181 +0x154
created by github.com/hashicorp/go-plugin.newGRPCStdioServer
        github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:37 +0x10c

goroutine 11 [IO wait, 18 minutes]:
internal/poll.runtime_pollWait(0x12c728170, 0x72)
        runtime/netpoll.go:305 +0xa0
internal/poll.(*pollDesc).wait(0x140001b6720?, 0x14000536000?, 0x1)
        internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140001b6720, {0x14000536000, 0x1000, 0x1000})
        internal/poll/fd_unix.go:167 +0x1e0
os.(*File).read(...)
        os/file_posix.go:31
os.(*File).Read(0x1400000e9a0, {0x14000536000?, 0x400?, 0x10371c7a0?})
        os/file.go:119 +0x5c
bufio.(*Reader).Read(0x14000086730, {0x14000534400, 0x400, 0x0?})
        bufio/bufio.go:237 +0x1e0
github.com/hashicorp/go-plugin.copyChan({0x103a1e178, 0x1400026a7e0}, 0x0?, {0x1039fcb60?, 0x1400000e9a0?})
        github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:181 +0x154
created by github.com/hashicorp/go-plugin.newGRPCStdioServer
        github.com/hashicorp/go-plugin@v1.4.3/grpc_stdio.go:38 +0x198

goroutine 37 [syscall, 18 minutes]:

Error: The terraform-provider-tanzu-mission-control_v1.4.4 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Reproduction steps

Unknown

I applied the terraform module several times in clean environments and I got this error only once source code is available here https://github.com/giovannibaratta/vmware-tanzu-training/tree/main/terraform/stages/50-tmc

Expected behavior

Terraform should not crash

Additional context

I am using TMC self managed v1.2 MacBook M1 (Terraform v1.7.4 on darwin_arm64)

github-actions[bot] commented 1 week ago

'Marking this issue as stale due to inactivity. This helps us focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!'