Connecting to Linode LKE using credentials errors

[superherointj]

Can't connect to Linode LKE using credentials (instead of kubeconfig file).

• Terraform version: v0.14.8
• Kubernetes Alpha Provider version: v0.3.2
• Kubernetes version: v1.20.4
• Terraform Cloud: Yes.

Brief

I'm configuring kubernetes-alpha provider credentials as:

provider "kubernetes-alpha" {
  host                   = yamldecode(base64decode(linode_lke_cluster.myapp-lke.kubeconfig)).clusters[0].cluster.server
  token                  = yamldecode(base64decode(linode_lke_cluster.myapp-lke.kubeconfig)).users[0].user.token
  cluster_ca_certificate = base64decode(yamldecode(base64decode(linode_lke_cluster.myapp-lke.kubeconfig)).clusters[0].cluster.certificate-authority-data)
}

But won't work. It errors as: Error: Failed to construct REST client

Demo at:

https://github.com/superherointj/tfprovider-kubernetes-alpha-linode-demo Thanks for having a look.

Steps to reproduce:

1. $ git clone https://github.com/superherointj/tfprovider-kubernetes-alpha-linode-demo.git
2. $ cd tfprovider-kubernetes-alpha-linode-demo
3. Configure backend "remote" in main.tf to your Terraform Cloud organization & workspace.
4. Set linode_token variable to your valid Linode token either at environment variable or Terraform Cloud's workspace variables (I use this).
5. $ terraform init
6. $ terraform plan

Then, terraform plan exits as:

Error: Failed to construct REST client

on samples.tf line 11, in resource "kubernetes_manifest" "test-configmap": 11: resource "kubernetes_manifest" "test-configmap" {

cannot create REST client: no client config

Notes

• Kubernetes-Alpha provider works fine when using config_path = "./kubeconfig.yaml" (file directly). But kubeconfig has no client-certificate-data or client-key-data.

• Kubernetes (official/standard) provider works perfectly with only host, cluster_ca_certificate, token from Linode's LKE. But kubernetes-alpha won't.

• Linode's LKE resource outputs a hashed kube_config. Requiring to base64decode kubeconfig before yamldecode.

• Sample Linode's LKE kubeconfig file: https://pastebin.com/8LWX6Cwz

• In LKE's kubeconfig client-certificate-data and client-key-data don't exist. There is only certificate-authority-data, token, server (host). How to fill in client_certificate and client_key?

To pass test

1. terraform plan won't error.
2. After terraform apply querying kubernetes cluster should return:
   • namespace demo_namespace => Kubernetes provider works.
   • ConfigMap test-config => Kubernetes-Alpha provider works.

Questions

• Should kubernetes-alpha provider be able to connect to Linode LKE with same credentials as official kubernetes provider does?
• How to fix configuration for kubernetes-alpha provider to connect to Linode LKE using credentials?

Thanks.

[alexsomesan]

Thanks for the very comprehensive reproduction material! I wish all issues would be this clearly documented. It makes it really easy to pinpoint the issue in no time.

Unfortunately, this provider doesn't support creating the cluster on which it operates in the same apply operation. In other words the cluster has to already be operational and available at plan time when using this provider. The reason for this is that this provider needs to make API calls to the cluster during the plan phase and the linode_lke_cluster resource has not yet been created at that point (it will have been created during the upcoming apply) and thus the client cert and key attributes don't yet have values and the alpha provider correctly reports "no client config" (yet).

By contrast, the original Kubernetes provider does need to make API calls at plan time and that is why it (sometimes) works in this kind of setup. However, Terraform itself doesn't guarantee reliable operation in this type of scenario where provider configuration attributes are set from attributes of other resources and those resources are not yet created. It's documented here.

I would suggest you to break off the provisioning of the LKE cluster in a separate apply step and use a remote state datasource to reference it's state so you can collect the configuration attributes when configuring this provider. Alternatively, you can also keep everything in one configuration like in your example, but create the LKE cluster in a partial apply using this command:

terraform apply -target linode_lke_cluster.demo-lke

Once that's done, you should be able to just do a normal terraform plan and see the plan for the kubernetes_manifest resources.

I hope that clarifies the issue well enough. Let me know if you have further questions or need more information.

[superherointj]

Thanks for your equally comprehensive answer. It was very helpful. :)

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!