Detect when running the launcher against GKE

[marccarre]

Problem

It is easy to accidentally select the incorrect environment, run the curl | sh command without --gke, and then experience errors like the below one:

$ curl -Ls https://get.dev.weave.works | sh -s -- --token=...
Downloading the Weave Cloud installer...  
Preparing for Weave Cloud setup
Checking kubectl & kubernetes versions
Connecting cluster to "gke-with-gpus" (id: gentle-pine-70) on Weave Cloud
Installing Weave Cloud agents on gke_gke-integration_us-central1-a_gke-with-gpus at https://35.184.236.64
Performing a check of the Kubernetes installation setup.
There was an error applying the agent: Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
Error from server (Forbidden): clusterroles.rbac.authorization.k8s.io "weave-agent" is forbidden: attempt to grant extra privileges: [PolicyRule{Resources:["*"], APIGroups:["*"], Verbs:["*"]} PolicyRule{NonResourceURLs:["*"], Verbs:["*"]}] user=&{marc@weave.works  [system:authenticated] map[authenticator:[GKE]]} ownerrules=[PolicyRule{Resources:["selfsubjectaccessreviews" "selfsubjectrulesreviews"], APIGroups:["authorization.k8s.io"], Verbs:["create"]} PolicyRule{NonResourceURLs:["/api" "/api/*" "/apis" "/apis/*" "/healthz" "/swagger-2.0.0.pb-v1" "/swagger.json" "/swaggerapi" "/swaggerapi/*" "/version"], Verbs:["get"]}] ruleResolutionErrors=[]
Full output:
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
namespace "weave" configured
serviceaccount "weave-agent" created
clusterrolebinding.rbac.authorization.k8s.io "weave-agent" created
deployment.extensions "weave-agent" created
Error from server (Forbidden): clusterroles.rbac.authorization.k8s.io "weave-agent" is forbidden: attempt to grant extra privileges: [PolicyRule{Resources:["*"], APIGroups:["*"], Verbs:["*"]} PolicyRule{NonResourceURLs:["*"], Verbs:["*"]}] user=&{marc@weave.works  [system:authenticated] map[authenticator:[GKE]]} ownerrules=[PolicyRule{Resources:["selfsubjectaccessreviews" "selfsubjectrulesreviews"], APIGroups:["authorization.k8s.io"], Verbs:["create"]} PolicyRule{NonResourceURLs:["/api" "/api/*" "/apis" "/apis/*" "/healthz" "/swagger-2.0.0.pb-v1" "/swagger.json" "/swaggerapi" "/swaggerapi/*" "/version"], Verbs:["get"]}] ruleResolutionErrors=[]
Rolling back cluster changes

Expected behaviour

• Automatically detect when running the launcher against GKE, and then
• Automatically create the corresponding clusterrolebinding / run with --gke.

[rade]

Automatically detect when running the launcher against GKE

How would we do this?

It would be really neat since it allows us to eliminate the env choice in the setup flow.

[dlespiau]

A few things looking at socks-shop:

• the nodes are called gke-sock-shop-default-pool-9652982b-fvfn
• kubernetes_build_info has a gitversion label with v1.9.6-gke.1
• The kube-proxy pod names have gke in them (seems to be the node name) kube-proxy-gke-sock-shop-default-pool-9652982b-7pbs"

[dlespiau]

We might be able to use a number of heuristics to check that's a GKE cluster, eg.

1. check API version

   $ kubectl version
   Server Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.6-gke.1", GitCommit:"cb151369f60073317da686a6ce7de36abe2bda8d", GitTreeState:"clean", BuildDate:"2018-04-07T22:06:59Z", GoVersion:"go1.9.3b4", Compiler:"gc", Platform:"linux/amd64"}

2. check the kubectl context name, currently gcloud container clusters get-credential always encode "gke" in the context name (of course the user can edit their kubeconfig).

   $ kubectl config current-context
   gke_sock-shop-stagingeurope-west2-b_sock-shop

[errordeveloper]

I think version string is a good one, and actually, yes nodes names seem to begin with gke-, but it'd be good to additionally check cloud.google.com/gke-nodepool node label, there is always at least one nodepool (otherwise there would be no nodes).

[rade]

instead of the context name, could we check the cluster name within the context, or is that user-settable?

- context:
    cluster: gke_sock-shop-staging_europe-west2-b_sock-shop
    user: gke_sock-shop-staging_europe-west2-b_sock-shop
  name: gke_sock-shop-staging_europe-west2-b_sock-shop

combined with the fact that the auth provider for the user is gcp.

[lilic]

If the current context auth-provider.name: gcp we can then query the metadata for more info? https://cloud.google.com/compute/docs/storing-retrieving-metadata

[marccarre]

An alternative we haven't discussed yet could be to have the end-users log in with their GCP account, and have the OAuth token sent to the launcher in some way. That would allow us to set things up transparently like we currently do for users coming from GCP Cloud Launcher.