edwardmedia
commented
10 months ago @gracjanborowiak can you share the config and repro steps?
Open gracjanborowiak opened 10 months ago
edwardmedia
commented
10 months ago @gracjanborowiak can you share the config and repro steps?
gracjanborowiak
commented
10 months ago very basic config.
steps i took:
dns firewall vpc vms negs
after some time and making changes in those files i got that error. i was not able to fix the terraform. i had to start over from different location with new state files.
hao-nan-li
commented
10 months ago Taking a look into this, any particular reason your negs file has .tff extension?
Looks like the error has something to do with google_compute_network_endpoints.
for instance = google_compute_instance.test-vm2.id, is that instance fully created before you run negs?
gracjanborowiak
commented
10 months ago hi,
tried to test alb with negs and in fact I was creating infra step by step.
indeed it died during neg creation.
tff was to remove neg from deployment for verification . it didn't work.
i removed all infra and started with new state. it worked like this.
I might be able to retrieve broken state file if you want.
czw., 18 sty 2024, 18:40 użytkownik hao-nan-li @.***> napisał:
Taking a look into this, any particular reason your negs file has .tff extension?
Looks like the error has something to do with google_compute_network_endpoints. for instance = google_compute_instance.test-vm2.id, is that instance fully created before you run negs?
— Reply to this email directly, view it on GitHub https://github.com/hashicorp/terraform-provider-google/issues/17018#issuecomment-1898932656, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHLO7XVL7QSWNSJHVLMQXKDYPFM7JAVCNFSM6AAAAABB5WBKTGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJYHEZTENRVGY . You are receiving this because you were mentioned.Message ID: @.***>
ggtisc
commented
8 months ago Hi @gracjanborowiak! Does this issue is related with this other one https://github.com/hashicorp/terraform-provider-google/issues/17286 ?
alxdrl
commented
5 months ago Just hit this today
Can repro with refresh + targeting:
terraform refresh -target='module.test.module.lb.google_compute_network_endpoints.all["ep1"]'
Produces the following log
2024/06/27 14:47:39 [DEBUG] Waiting for state to become: [success]
2024/06/27 14:47:39 [DEBUG] Retry Transport: starting RoundTrip retry loop
2024/06/27 14:47:39 [DEBUG] Retry Transport: request attempt 0
2024/06/27 14:47:39 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
POST /compute/v1/projects/<projectId>/zones/<zone>/networkEndpointGroups/<neg>/listNetworkEndpoints?alt=json HTTP/1.1
Host: compute.googleapis.com
User-Agent: Terraform/1.8.5 (+https://www.terraform.io) Terraform-Plugin-SDK/2.33.0 terraform-provider-google/5.35.0
Content-Length: 0
Content-Type: application/json
Accept-Encoding: gzip
-----------------------------------------------------
2024/06/27 14:47:40 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 12:47:40 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0
{
"kind": "compute#networkEndpointGroupsListNetworkEndpoints",
"id": "projects/<projectId>/zones/<zone>/networkEndpointGroups/<neg>/listNetworkEndpoints"
}
-----------------------------------------------------
2024/06/27 14:47:40 [DEBUG] Retry Transport: Stopping retries, last request was successful
2024/06/27 14:47:40 [DEBUG] Retry Transport: Returning after 1 attempts
panic: interface conversion: interface {} is nil, not []interface {}
goroutine 127 [running]:
github.com/hashicorp/terraform-provider-google/google/services/compute.resourceComputeNetworkEndpointsDecoder(0xc000341500?, {0x3e1cd00?, 0xc000341500}, 0xc001869e30?)
github.com/hashicorp/terraform-provider-google/google/services/compute/resource_compute_network_endpoints.go:768 +0x565
github.com/hashicorp/terraform-provider-google/google/services/compute.resourceComputeNetworkEndpointsRead(0xc0016f4e00, {0x3e1cd00?, 0xc000341500?})
github.com/hashicorp/terraform-provider-google/google/services/compute/resource_compute_network_endpoints.go:347 +0x40e
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0x45c0398?, {0x45c0398?, 0xc000e6eb70?}, 0xd?, {0x3e1cd00?, 0xc000341500?})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.33.0/helper/schema/resource.go:783 +0x163
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc000917960, {0x45c0398, 0xc000e6eb70}, 0xc000c22270, {0x3e1cd00, 0xc000341500})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.33.0/helper/schema/resource.go:1089 +0x552
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc00012c8a0, {0x45c0398?, 0xc000e6ea80?}, 0xc0014725c0)
github.com/hashicorp/terraform-plugin-sdk/v2@v2.33.0/helper/schema/grpc_provider.go:667 +0x48a
github.com/hashicorp/terraform-plugin-mux/tf5muxserver.(*muxServer).ReadResource(0x45c03d0?, {0x45c0398?, 0xc000e6e630?}, 0xc0014725c0)
github.com/hashicorp/terraform-plugin-mux@v0.15.0/tf5muxserver/mux_server_ReadResource.go:35 +0x193
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc0005225a0, {0x45c0398?, 0xc001009cb0?}, 0xc00145eba0)
github.com/hashicorp/terraform-plugin-go@v0.22.1/tfprotov5/tf5server/server.go:775 +0x4c3
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0x3dc46e0?, 0xc0005225a0}, {0x45c0398, 0xc001009cb0}, 0xc0016f4b80, 0x0)
github.com/hashicorp/terraform-plugin-go@v0.22.1/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:482 +0x169
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000215e00, {0x45c0398, 0xc001009c20}, {0x45cb330, 0xc00051fb00}, 0xc001305b00, 0xc000fc73b0, 0x61920a8, 0x0)
google.golang.org/grpc@v1.63.2/server.go:1369 +0xe23
google.golang.org/grpc.(*Server).handleStream(0xc000215e00, {0x45cb330, 0xc00051fb00}, 0xc001305b00)
google.golang.org/grpc@v1.63.2/server.go:1780 +0x1016
google.golang.org/grpc.(*Server).serveStreams.func2.1()
google.golang.org/grpc@v1.63.2/server.go:1019 +0x8b
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 26
google.golang.org/grpc@v1.63.2/server.go:1030 +0x135
aaliddell
commented
2 months ago Also just hit this.
I created a NEG and added an endpoint with TF. There was then a conflict as I added the instance via its self_link rather than name, which would fail to delete the endpoint due to having null port that got coerced to 0 (this is a separate bug btw!). So I tried manually deleting the endpoint within the group, after which I got the error seen above. Only solution then was to delete the entire NEG and restart.
So perhaps the error occurs when the state file says there should be endpoints but the API returns none?
ggtisc
commented
2 months ago Please confirm us if the version of terraform you have matches with your OS. You could achieve this with a terraform -version looking for the OS
aaliddell
commented
2 months ago Yep, correct executable.
The answer is there in the error message from @alxdrl: the response object for listNetworkEndpoints has no items property when there are no endpoints, rather than the expected empty list. In resourceComputeNetworkEndpointsDecoder the items property is blindly cast without checking if it is nil:
This response violates the API spec, but the easier fix here (over getting Google to fix) is to check if the items property exists and if not assume an empty list.
ggtisc
commented
2 months ago Forwarding because many users are presenting the same issue
Stack trace from the terraform-provider-google_v5.12.0_x5 plugin:
panic: interface conversion: interface {} is nil, not []interface {}
goroutine 190 [running]: github.com/hashicorp/terraform-provider-google/google/services/compute.resourceComputeNetworkEndpointsDecoder(0xc0002b0a80?, {0x3b93260?, 0xc0002b0a80}, 0xc001b74cf0?) github.com/hashicorp/terraform-provider-google/google/services/compute/resource_compute_network_endpoints.go:758 +0x585 github.com/hashicorp/terraform-provider-google/google/services/compute.resourceComputeNetworkEndpointsRead(0xc0020c8a00, {0x3b93260?, 0xc0002b0a80?}) github.com/hashicorp/terraform-provider-google/google/services/compute/resource_compute_network_endpoints.go:342 +0x3f5 github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(Resource).read(0x42afb78?, {0x42afb78?, 0xc0012a0180?}, 0xd?, {0x3b93260?, 0xc0002b0a80?}) github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.0/helper/schema/resource.go:712 +0x178 github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(Resource).RefreshWithoutUpgrade(0xc000b5e0e0, {0x42afb78, 0xc0012a0180}, 0xc000964f70, {0x3b93260, 0xc0002b0a80}) github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.0/helper/schema/resource.go:1015 +0x585 github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(GRPCProviderServer).ReadResource(0xc000f1e648, {0x42afb78?, 0xc0012a0060?}, 0xc001a0cf40) github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.0/helper/schema/grpc_provider.go:613 +0x4a5 github.com/hashicorp/terraform-plugin-mux/tf5muxserver.muxServer.ReadResource({0xc000ecc330, 0xc000ecc390, {0xc00156c1a0, 0x2, 0x2}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...) github.com/hashicorp/terraform-plugin-mux@v0.8.0/tf5muxserver/mux_server_ReadResource.go:26 +0x102 github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(server).ReadResource(0xc0011ebf40, {0x42afb78?, 0xc000d71620?}, 0xc000649140) github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/tf5server/server.go:748 +0x4b1 github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0x3b059c0?, 0xc0011ebf40}, {0x42afb78, 0xc000d71620}, 0xc0020c8780, 0x0) github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:349 +0x170 google.golang.org/grpc.(Server).processUnaryRPC(0xc0010d8f00, {0x42afb78, 0xc000d71590}, {0x42b9318, 0xc0014b3520}, 0xc001128ea0, 0xc001646360, 0x5bc4350, 0x0) google.golang.org/grpc@v1.60.0/server.go:1372 +0xe49 google.golang.org/grpc.(Server).handleStream(0xc0010d8f00, {0x42b9318, 0xc0014b3520}, 0xc001128ea0) google.golang.org/grpc@v1.60.0/server.go:1783 +0x1031 google.golang.org/grpc.(Server).serveStreams.func2.1() google.golang.org/grpc@v1.60.0/server.go:1016 +0x68 created by google.golang.org/grpc.(Server).serveStreams.func2 google.golang.org/grpc@v1.60.0/server.go:1027 +0x12e
Error: The terraform-provider-google_v5.12.0_x5 plugin crashed!
b/371060894