Random crash with big quantity of ressources

Fran-E commented 3 months ago

Prerequisites

[no] Have you tested the operation in the API directly?
[yes] Do you have the latest Terraform provider version?
[yes] Review the compatibility matrix before opening an issue.

Describe the bug Randomly there is a crash sometime during the plan, sometime during the apply. This looks to be in relation with the quantity of creation to do. To minimise I reduce this meraki_requests_per_second but this is always exist.

Expected behavior Plan and apply have to finish properly

Screenshots here output I have when this is happening :

....... ╷ │ Error: Request cancelled │ │ The plugin6.(GRPCProvider).UpgradeResourceState request was cancelled. ╵ ╷ │ Error: Request cancelled │ │ The plugin6.(GRPCProvider).UpgradeResourceState request was cancelled. ╵ ╷ │ Error: Failure when executing GetNetworkGroupPolicy │ │ with meraki_networks_group_policies.policynetatmo["SAINTLEONARDCB-MX"], │ on grouppolicy.tf line 1, in resource "meraki_networks_group_policies" "policynetatmo": │ 1: resource "meraki_networks_group_policies" "policynetatmo" { │ │ error with operation GetNetworkGroupPolicy ╵ ╷ │ Error: Failure when executing GetNetworkGroupPolicy │ │ with meraki_networks_group_policies.policynetatmo["ROUENCB-MX"], │ on grouppolicy.tf line 1, in resource "meraki_networks_group_policies" "policynetatmo": │ 1: resource "meraki_networks_group_policies" "policynetatmo" { │ │ error with operation GetNetworkGroupPolicy ╵ ╷ │ Error: Failure when executing GetNetworkGroupPolicy │ │ with meraki_networks_group_policies.policynetatmo["COURTENAYBB-MX"], │ on grouppolicy.tf line 1, in resource "meraki_networks_group_policies" "policynetatmo": │ 1: resource "meraki_networks_group_policies" "policynetatmo" { │ │ error with operation GetNetworkGroupPolicy ╵ ╷ │ Error: Failure when executing GetNetworkGroupPolicy │ │ with meraki_networks_group_policies.policynetatmo["NANTERRECB-MX"], │ on grouppolicy.tf line 1, in resource "meraki_networks_group_policies" "policynetatmo": │ 1: resource "meraki_networks_group_policies" "policynetatmo" { │ │ error with operation GetNetworkGroupPolicy ╵

Stack trace from the terraform-provider-meraki_v0.2.9-alpha.exe plugin:

fatal error: concurrent map writes

goroutine 314 [running]: reflect.mapassign_faststr0(0x14634c0, 0x143a260?, {0xc00087ac50?, 0x1a19188?}, 0x13d2580?) runtime/map.go:1376 +0x25 reflect.mapassign_faststr(0x143a260?, 0xc0003925f0?, {0xc00087ac50, 0x6}, 0x19?) reflect/value.go:3837 +0xa9 reflect.Value.SetMapIndex({0x14634c0?, 0x2296b58?, 0x1fe?}, {0x13d2580, 0xc000392620, 0x98}, {0x143a260, 0xc0003925f0, 0x194}) reflect/value.go:2402 +0x225 encoding/json.(decodeState).object(0xc00099eb40, {0x13a6e40?, 0x2296b58?, 0x22993a0?}) encoding/json/decode.go:797 +0x13c5 encoding/json.(decodeState).value(0xc00099eb40, {0x13a6e40?, 0x2296b58?, 0x275f2be0598?}) encoding/json/decode.go:374 +0x3e encoding/json.(decodeState).unmarshal(0xc00099eb40, {0x13a6e40?, 0x2296b58?}) encoding/json/decode.go:181 +0x133 encoding/json.Unmarshal({0xc00093e200, 0x37, 0x200}, {0x13a6e40, 0x2296b58}) encoding/json/decode.go:108 +0x111 github.com/go-resty/resty/v2.Unmarshalc(0xc0002e2f00, {0xc0006c4160, 0x1f}, {0xc00093e200, 0x37, 0x200}, {0x13a6e40, 0x2296b58}) github.com/go-resty/resty/v2@v2.13.1/util.go:120 +0xc5 github.com/go-resty/resty/v2.parseResponseBody(0xc0002e2f00, 0xc000d21b30) github.com/go-resty/resty/v2@v2.13.1/middleware.go:405 +0x40e github.com/go-resty/resty/v2.(Client).execute(0xc0002e2f00, 0xc000a52000) github.com/go-resty/resty/v2@v2.13.1/client.go:1246 +0xdf9 github.com/go-resty/resty/v2.(Request).Execute.func2() github.com/go-resty/resty/v2@v2.13.1/request.go:947 +0x8a github.com/go-resty/resty/v2.Backoff(0xc000c5efc0, {0xc00065ef90, 0x6, 0xc0002e0180?}) github.com/go-resty/resty/v2@v2.13.1/retry.go:115 +0x16c github.com/go-resty/resty/v2.(Request).Execute(0xc000a52000, {0x16d67b9?, 0x16d87d0?}, {0xc00001ac00, 0x37}) github.com/go-resty/resty/v2@v2.13.1/request.go:941 +0x5f4 github.com/go-resty/resty/v2.(Request).Get(...) github.com/go-resty/resty/v2@v2.13.1/request.go:852 github.com/meraki/dashboard-api-go/v3/sdk.(NetworksService).GetNetworkGroupPolicy(0xc0002dda00, {0xc0002c87e0, 0x14}, {0xc00057c520, 0x3}) github.com/meraki/dashboard-api-go/v3@v3.0.7/sdk/networks.go:4934 +0x57e github.com/cisco-open/terraform-provider-meraki/internal/provider.(NetworksGroupPoliciesResource).Read(0xc0006d0000, {0x19e31e8, 0xc000e8ca50}, {{{{0x19eae70, 0xc000c099e0}, {0x146b920, 0xc000a29470}}, {0x19fd140, 0xc000083130}}, 0xc0002e0168, ...}, ...) github.com/cisco-open/terraform-provider-meraki/internal/provider/resource_meraki_networks_group_policies.go:1064 +0x31f github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(Server).ReadResource(0xc0002de820, {0x19e31e8, 0xc000e8ca50}, 0xc000e8cae0, 0xc000c5f6d0) github.com/hashicorp/terraform-plugin-framework@v1.7.0/internal/fwserver/server_readresource.go:101 +0x64e github.com/hashicorp/terraform-plugin-framework/internal/proto6server.(Server).ReadResource(0xc0002de820, {0x19e31e8?, 0xc000e8c930?}, 0xc0003a8bc0) github.com/hashicorp/terraform-plugin-framework@v1.7.0/internal/proto6server/server_readresource.go:55 +0x3e5 github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(server).ReadResource(0xc0002c20a0, {0x19e31e8?, 0xc000e8c0c0?}, 0xc0000b2000) github.com/hashicorp/terraform-plugin-go@v0.23.0/tfprotov6/tf6server/server.go:784 +0x309 github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ReadResource_Handler({0x1696660?, 0xc0002c20a0}, {0x19e31e8, 0xc000e8c0c0}, 0xc0005d2580, 0x0) github.com/hashicorp/terraform-plugin-go@v0.23.0/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:482 +0x169 google.golang.org/grpc.(Server).processUnaryRPC(0xc0002e8000, {0x19e31e8, 0xc000e8c030}, {0x19fb5b8, 0xc000424000}, 0xc0001f8000, 0xc000215d70, 0x228d3a8, 0x0) google.golang.org/grpc@v1.63.2/server.go:1369 +0xe23 google.golang.org/grpc.(Server).handleStream(0xc0002e8000, {0x19fb5b8, 0xc000424000}, 0xc0001f8000) google.golang.org/grpc@v1.63.2/server.go:1780 +0x1016 google.golang.org/grpc.(Server).serveStreams.func2.1() google.golang.org/grpc@v1.63.2/server.go:1019 +0x8b created by google.golang.org/grpc.(Server).serveStreams.func2 in goroutine 26 google.golang.org/grpc@v1.63.2/server.go:1030 +0x135

goroutine 1 [select]: github.com/hashicorp/go-plugin.Serve(0xc000081d40) github.com/hashicorp/go-plugin@v1.6.0/server.go:503 +0x166d github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.Serve({0x1708c01, 0x28}, 0xc000163080, {0x0, 0x0, 0x0}) github.com/hashicorp/terraform-plugin-go@v0.23.0/tfprotov6/tf6server/server.go:316 +0xc37 github.com/hashicorp/terraform-plugin-framework/providerserver.Serve({0x19e31b0?, 0x22ee340?}, 0xc000009248, {{0x1708c01?, 0x5?}, 0xee?, 0x45?}) github.com/hashicorp/terraform-plugin-framework@v1.7.0/providerserver/providerserver.go:114 +0x271 main.main() github.com/cisco-open/terraform-provider-meraki/main.go:60 +0x12c

goroutine 18 [select]: github.com/hashicorp/go-plugin.(gRPCBrokerServer).Recv(0x0?) github.com/hashicorp/go-plugin@v1.6.0/grpc_broker.go:128 +0x65 github.com/hashicorp/go-plugin.(GRPCBroker).Run(0xc0002ea0b0) github.com/hashicorp/go-plugin@v1.6.0/grpc_broker.go:582 +0x4f created by github.com/hashicorp/go-plugin.(*GRPCServer).Init in goroutine 1 github.com/hashicorp/go-plugin@v1.6.0/grpc_server.go:91 +0x59f

goroutine 19 [syscall, locked to thread]: syscall.SyscallN(0x0?, {0xc00007bc40?, 0x0?, 0x0?}) runtime/syscall_windows.go:544 +0x107 syscall.Syscall6(0x0?, 0x0?, 0x0?, 0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/syscall_windows.go:488 +0x4a syscall.readFile(0x0?, {0xc000121000?, 0x1000, 0x0?}, 0x0?, 0x800000?) syscall/zsyscall_windows.go:1024 +0x8e syscall.ReadFile(...) syscall/syscall_windows.go:438 syscall.Read(0xc0002e2000?, {0xc000121000?, 0x0?, 0x0?}) syscall/syscall_windows.go:417 +0x2d internal/poll.(FD).Read(0xc0002e2000, {0xc000121000, 0x1000, 0x1000}) internal/poll/fd_windows.go:422 +0x1c5 os.(File).read(...) os/file_posix.go:29 os.(File).Read(0xc0002e0000, {0xc000121000?, 0x400?, 0x1403a40?}) os/file.go:118 +0x52 bufio.(Reader).Read(0xc00007bf40, {0xc000086800, 0x400, 0x0?}) bufio/bufio.go:244 +0x197 github.com/hashicorp/go-plugin.copyChan({0x1a11ec0, 0xc00017e820}, 0x0?, {0x19d9100?, 0xc0002e0000?}) github.com/hashicorp/go-plugin@v1.6.0/grpc_stdio.go:184 +0x1f6 created by github.com/hashicorp/go-plugin.newGRPCStdioServer in goroutine 1 github.com/hashicorp/go-plugin@v1.6.0/grpc_stdio.go:40 +0xed

goroutine 20 [syscall, locked to thread]: syscall.SyscallN(0x0?, {0xc0002f1c40?, 0x0?, 0x0?}) runtime/syscall_windows.go:544 +0x107 syscall.Syscall6(0x275f7f120d8?, 0x52b680?, 0x22b05a0?, 0x1?, 0xc0002f1c80?, 0x41?, 0xc0002f1cf0?, 0x519f68?) runtime/syscall_windows.go:488 +0x4a syscall.readFile(0x519b73?, {0xc000306000?, 0x1000, 0x0?}, 0x0?, 0x800000?) syscall/zsyscall_windows.go:1024 +0x8e

Error: The terraform-provider-meraki_v0.2.9-alpha.exe plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely helpful if you could report the crash with the plugin's maintainers so that it can be fixed. The output above should help diagnose the issue.

Environment (please complete the following information):

Meraki Dashboard version: ?
Terraform version: 1.9.2
Meraki provider version: 0.2.9-alpha
OS Version: Windows 11

Thank you for your job here, I was waiting terraform provider for Meraki since long time :-)

fmunozmiranda commented 3 months ago

Hi @Fran-E . Please try it lowering the number of requests per second. https://registry.terraform.io/providers/cisco-open/meraki/latest/docs#meraki_requests_per_second

Fran-E commented 3 months ago

Hi @Fran-E . Please try it lowering the number of requests per second. https://registry.terraform.io/providers/cisco-open/meraki/latest/docs#meraki_requests_per_second

Hello,

thanks, this is my workaround for now. When I put 2 it's more stable.

Br,

jon-humphries commented 3 months ago

@Fran-E Try also using terraform apply -parallelism=1 when you debug without this feature you will see in logs that Meraki API is advising you to lower the requests.

fmunozmiranda commented 3 months ago

thank you for your feedback, closing this issue.

joekohlsdorf commented 2 weeks ago

@fmunozmiranda Please reopen this issue until we have a proper fix in the provider.

It is not resolved, users of the provider shouldn't need to search for workarounds in a closed GitHub issue.

I can reproduce this very consistently with the following minimal code:

resource "meraki_devices_switch_ports" "this_switch_port" {
  serial = "XXX-XXX-XXX"
  count = 40
  port_id = count.index + 1
  name = "test"
}

cisco-open / terraform-provider-meraki

Random crash with big quantity of ressources #119