Add recommendations on GKE authentication / K8S usage

[rileykarson]

It's common for TPG users to want to authenticate with K8S in clusters provisioned through Terraform.

It's an open issue in the CFT GKE module to generate a kubeconfig file; we've provided a reference implementation of Terraform-native OpenId authentication in the Gruntwork GKE module; this repo has had a PR adding a kubeconfig output to google_container_cluster; and I've got another example of Terraform-native OpenId authentication kicking around in a datasource that doesn't look GKE-related at all.

I'm not in love with solutions that generate kubeconfig files, because they're generally supporting cert-based auth or password-based auth. Those methods are against the recommendations at https://cloud.google.com/kubernetes-engine/docs/how-to/hardening-your-cluster#restrict_authn_methods. Instead, I'd like to promote the appropriate OpenId-based auth methods depending on the user's use case.

I think we can resolve this issue with these steps, but the implementer can make the final decision.

• Stick the Terraform-native auth example somewhere easier to discover, link it from the GKE docs

• Properly recommend using gcloud to generate kubeconfig entries when using K8S tooling like kubectl (see https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl for reference)

• Investigate whether we can generate kubeconfig files with access tokens for users who don't want to use gcloud. If it's possible, provide details on how to do so.

[rileykarson]

@morgante: If you or anyone from CFT has thoughts, let us know.

[morgante]

I agree this is definitely a pain point, but also agree with avoiding legacy cert or password-based auth.

My preference would be:

1. Stick the Terraform-native auth example somewhere easier to discover, link it from the GKE docs.

Could we even package this as a small GKE submodule?

2. Investigate whether we can generate kubeconfig files with access tokens for users who don't want to use gcloud. If it's possible, provide details on how to do so.

Based on some non-Terraform work, I do think this is possible. Ideally we could make this an additional output of the module.

[rileykarson]

1. We've been talking about adding a GKE section to the provider's Guides, and link from the GKE resources. We'd cover auth + probably some details on node pools and VPC-native clusters. Given that, would you still like to see a submodule?

2. I believe it's possible, but it was unclear what the format was to me when I last looked. I think it would make sense as a datasource.

[morgante]

We've been talking about adding a GKE section to the provider's Guides, and link from the GKE resources. We'd cover auth + probably some details on node pools and VPC-native clusters. Given that, would you still like to see a submodule?

I think it'd still be better as a submodule. Having to copy a bunch of code from the provider docs is fine (and better than having to develop it yourself), but still not as easy as invoking a submodule which automatically configures the Kubernetes provider for me.

[rileykarson]

I tried making a kubeconfig again, we can directly feed a token into one. https://www.terraform.io/docs/providers/google/d/datasource_client_config.html#access_token can be used directly as the token. I'd imagine https://www.terraform.io/docs/providers/google/d/datasource_google_service_account_access_token.html#access_token works too, in that case.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!