search

GoogleCloudPlatform / traffic-director-grpc-bootstrap

Apache License 2.0
20 stars 18 forks source link

Generate Mesh ID for zonal/regional clusters #47

Closed arvindbr8 closed 10 months ago

arvindbr8 commented 1 year ago

For Proxyless Service Mesh usecase, the GKE cluster could be created to be either zonal or regional. Currently we only retrieve the zone from the metadata server. Which means using --generate-mesh-id-experimental would generate an incorrect mesh ID for a mesh deployed in a regional cluster.

The bootstrap generator is typically run in an init container for GKE workloads. We should find a way to figure out whether the deployment type of the cluster and we think the metadata server should be able to give us this information.

Currently the getZone method uses http://metadata.google.internal/computeMetadata/v1/instance/zone -- which retrieves the zone where the pod is located.

Potential metadata server APIs that could work for us:

  1. http://metadata.google.internal/computeMetadata/v1/instance/attributes/cluster-location
    • which retrieves the location -- (zone/region)
  2. http://metadata.google.internal/computeMetadata/v1/instance/attributes/kube-env
    • which retrieves all kube envs. We need to look for ZONE:

Before we proceed, we need a confirmation from the folks who maintain the "metadata server" which of these options are more reliable.

arvindbr8 commented 1 year ago

We heard back from the GKE MDS folks and seems like the http://metadata.google.internal/computeMetadata/v1/instance/attributes/cluster-location endpoint is the right one to use.

Luckily I had a "few" clusters of different configurations "lying" around in my GCP project (439293274322). So I got some testing done on a regional cluster and another zonal cluster... and it checks out.

cluster name: my-cluster-1 location: us-west1 (regional) pod: psm-grpc-client-bfd7bbbdf-69frh

$ curl --header 'Metadata-Flavor:Google'  http://metadata.google.internal/computeMetadata/v1/instance/attributes/cluster-location
us-west1

cluster name: arvindbright-test-cluster-fork8ntesting location: us-west1 (regional) pod: psm-grpc-server-56b959d689-ntt4k

$ curl --header 'Metadata-Flavor:Google'  http://metadata.google.internal/computeMetadata/v1/instance/attributes/cluster-location
us-west1-a