[Bug]: Some resources not available with `kubectl get ...` (Client.Timeout exceeded while awaiting headers)

[Bluesboy]

Is there an existing issue for this?

• [X] I have searched the existing issues

Affected Resource(s)

• compute.gcp.upbound.io/v1beta1 - Subnetwork

maybe some others

Resource MRs required to reproduce the bug

Subnetwork

Steps to Reproduce

apiVersion: compute.gcp.upbound.io/v1beta1
kind: Subnetwork
metadata:
  name: cloudrun-eu
  annotations:
    crossplane.io/external-name: cloudrun
    region: europe-west1
  labels:
    example.com/subnet: cloudrun
    example.com/region: europe-west1
spec:
  forProvider:
    ipCidrRange: 172.21.0.0/23
    network: "example-net"
    region: europe-west1

What happened?

I'm trying to deploy Cloud Run service with Direct VPC Access enabled. Working with fresh Crossplane installation. To enable VPC Access Cloud Run needs VPC subnetwork, so I'm trying to create Subnet with Crossplane. When there is no Subnetwork manifests present in the cluster kubectl get subnetworks gives me:

No resources found

which is fine, but if I apply any Subnetwork MR to the cluster kubectl get subnetworks returns error:

Error from server: conversion webhook for compute.gcp.upbound.io/v1beta1, Kind=Subnetwork failed: Post "https://upbound-provider-gcp-compute.crossplane-system.svc:9443/convert?timeout=30s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

resource itself is created, but I can't see its status and, which is more concerning, Argo CD can't determine it as well.

I turned on debug log for upbound-provider-gcp-compute and sometimes it's throwing bunch of errors like this

k8s.io/client-go@v0.29.4/tools/cache/reflector.go:229: failed to list *v1beta1.Subnetwork: subnetworks.compute.gcp.upbound.io is forbidden: User "system:serviceaccount:crossplane-system:upbound-provider-gcp-compute-dcb094af7952" cannot list resource "subnetworks" in API group "compute.gcp.upbound.io" at the cluster scope

k8s.io/client-go@v0.29.4/tools/cache/reflector.go:229: Failed to watch *v1beta1.Subnetwork: failed to list *v1beta1.Subnetwork: subnetworks.compute.gcp.upbound.io is forbidden: User "system:serviceaccount:crossplane-system:upbound-provider-gcp-compute-dcb094af7952" cannot list resource "subnetworks" in API group "compute.gcp.upbound.io" at the cluster scope

not only for subnets, actually all CRDs of the provider are mentioned

I have other GKE cluster for testing purposes in other GCP project and everything works completely fine there.

Relevant Error Output Snippet

No response

Crossplane Version

1.16.0

Provider Version

1.5.0

Kubernetes Version

v1.29.6-gke.1038001

Kubernetes Distribution

GKE

Additional Info

No response

[Bluesboy]

Sorted out the issue — for some reason GKE Control Plane was blocked by firewall on this particular cluster. Added firewall rule to allow ingress traffic from GKE Control Plane network to Crossplane webhook port (9443) in cluster network. Now everything seems fine. Though I still not understand why I haven't had this issue for other MRs (GlobalAddress for example always was fine)

[turkenf]

I'm glad your problem was solved. I didn't look at it in detail, but the answer to your question may be that there is only the v1beta1 version of the GlobalAddress resource, and there is no need for conversion webhook.