data.terraform_remote_state.name.outputs not picked up in provider configuration, causes panic or Error: rpc error: code = Unavailable desc = transport is closing

fredo838 commented 4 years ago

Terraform Version and Provider Version

Terraform v0.12.23

Kubernetes Version

Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.5", GitCommit:"e0fccafd69541e3750d460ba0f9743b90336f24f", GitTreeState:"clean", BuildDate:"2020-04-16T11:44:03Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.6-gke.11", GitCommit:"d20a4fc711a389212174a062bd295668089a110f", GitTreeState:"clean", BuildDate:"2020-06-15T17:56:39Z", GoVersion:"go1.13.9b4", Compiler:"gc", Platform:"linux/amd64"}

Error explained

terraform plan fails using following code block

data "google_client_config" "terraform_config" {
  provider = google
}

data "terraform_remote_state" "sd" {
  backend = "gcs"
  config = {
    bucket = var.state_bucket
    prefix = "shared"
  }
  workspace = "default"
}

provider "kubernetes" {
  load_config_file = false
  host = "https://${data.terraform_remote_state.sd.outputs.endpoint}"
  cluster_ca_certificate = base64decode(data.terraform_remote_state.sd.outputs.ca_certificate)
  token = data.google_client_config.terraform_config.access_token
}

provider "kubernetes-alpha" {
  host = "https://${data.terraform_remote_state.sd.outputs.endpoint}"
  cluster_ca_certificate = base64decode(data.terraform_remote_state.sd.outputs.ca_certificate)
  token = data.google_client_config.terraform_config.access_token
  //  config_path = "~/.kube/config"
}

resource "kubernetes_manifest" "test-configmap" {
  provider = kubernetes-alpha

  manifest = {
    "apiVersion" = "v1"
    "kind" = "ConfigMap"
    "metadata" = {
      "name" = "test-config"
      "namespace" = "default"
    }
    "data" = {
      "foo" = "bar"
    }
  }
}

when I replace the variables in the second to last block with their hard coded values:

provider "kubernetes-alpha" {
  host = "https://132.XXX.XXX.XXX"
  cluster_ca_certificate = base64decode("XXXXX...XXX")
  token = data.google_client_config.terraform_config.access_token
  //  config_path = "~/.kube/config"
}

terraform plan is working fine, same with uncommenting //config_path = "~/.kube/config". The issue is not present when using the "old" kubernetes provider (indicating this is not some kind of configuration issue but a bug.)

The Error I get is: Error: rpc error: code = Unavailable desc = transport is closing, indicating misconfigured credentials, or sometimes I get a panic error:

Error: rpc error: code = Unavailable desc = transport is closing

panic: value is unknown
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha: 
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha: goroutine 35 [running]:
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha: github.com/hashicorp/go-cty/cty.Value.AsString(0x1860ee0, 0xc000044fdb, 0x146a900, 0x23638c0, 0x1628b00, 0x4)
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha:        /home/runner/work/terraform-provider-kubernetes-alpha/terraform-provider-kubernetes-alpha/vendor/github.com/hashicorp/go-cty/cty/value_ops.go:1179 +0x16b
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha: github.com/hashicorp/terraform-provider-kubernetes-alpha/provider.(*RawProviderServer).PrepareProviderConfig(0x2363bf0, 0x1860a20, 0xc00019c540, 0xc00019cbd0, 0x2363bf0, 0xc00019c540, 0xc000478b78)
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha:        /home/runner/work/terraform-provider-kubernetes-alpha/terraform-provider-kubernetes-alpha/provider/server.go:105 +0xd96
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha: github.com/hashicorp/terraform-provider-kubernetes-alpha/tfplugin5._Provider_PrepareProviderConfig_Handler(0x1565b60, 0x2363bf0, 0x1860a20, 0xc00019c540, 0xc0002f01e0, 0x0, 0x1860a20, 0xc00019c540, 0xc00055a1c0, 0xd6)
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha:        /home/runner/work/terraform-provider-kubernetes-alpha/terraform-provider-kubernetes-alpha/tfplugin5/tfplugin5.pb.go:3064 +0x217
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha: google.golang.org/grpc.(*Server).processUnaryRPC(0xc000582600, 0x1872180, 0xc000482900, 0xc0000f4000, 0xc0000993e0, 0x2322038, 0x0, 0x0, 0x0)
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha:        /home/runner/work/terraform-provider-kubernetes-alpha/terraform-provider-kubernetes-alpha/vendor/google.golang.org/grpc/server.go:1024 +0x501
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha: google.golang.org/grpc.(*Server).handleStream(0xc000582600, 0x1872180, 0xc000482900, 0xc0000f4000, 0x0)
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha:        /home/runner/work/terraform-provider-kubernetes-alpha/terraform-provider-kubernetes-alpha/vendor/google.golang.org/grpc/server.go:1313 +0xd3d
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha: google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0001a2080, 0xc000582600, 0x1872180, 0xc000482900, 0xc0000f4000)
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha:        /home/runner/work/terraform-provider-kubernetes-alpha/terraform-provider-kubernetes-alpha/vendor/google.golang.org/grpc/server.go:722 +0xa1
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha: created by google.golang.org/grpc.(*Server).serveStreams.func1
2020-07-22T00:44:36.555+0200 [DEBUG] plugin.terraform-provider-kubernetes-alpha:        /home/runner/work/terraform-provider-kubernetes-alpha/terraform-provider-kubernetes-alpha/vendor/google.golang.org/grpc/server.go:720 +0xa1
2020-07-22T00:44:36.556+0200 [DEBUG] plugin: plugin process exited: path=/home/fred/Desktop/google-drive/avrotros-kee/terraform/env/.terraform/plugins/linux_amd64/terraform-provider-kubernetes-alpha pid=23932 error="exit status 2"
2020/07/22 00:44:36 [WARN] <root>: eval: *terraform.EvalValidateProvider, non-fatal err: rpc error: code = Unavailable desc = transport is closing
2020/07/22 00:44:36 [ERROR] <root>: eval: *terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing
2020/07/22 00:44:36 [ERROR] <root>: eval: *terraform.EvalOpFilter, err: rpc error: code = Unavailable desc = transport is closing
2020/07/22 00:44:36 [ERROR] <root>: eval: *terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing
2020/07/22 00:44:36 [TRACE] [walkValidate] Exiting eval tree: provider.kubernetes-alpha
2020/07/22 00:44:36 [TRACE] vertex "provider.kubernetes-alpha": visit complete

...
2020/07/22 00:44:36 [TRACE] vertex "provider.kubernetes (close)": visit complete
2020/07/22 00:44:36 [TRACE] dag/walk: upstream of "root" errored, so skipping
2020-07-22T00:44:36.794+0200 [DEBUG] plugin: plugin exited

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
A crash log has been placed at "crash.log" relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version. That
information is available on the first line of crash.log. You can also
get it by running 'terraform --version' on the command line.

SECURITY WARNING: the "crash.log" file that was created may contain 
sensitive information that must be redacted before it is safe to share 
on the issue tracker.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

I can provide a full log,

Steps to reproduct: 1) Create remote state 1 with an output 2) depend on it with terraform_remote_state in the provider block 3) terraform plan

I am guessing for some reason the values from data.terraform_remote_state are empty when the provider is initialized, which shouldn't happen (as the old kubernetes provider handles this case), I think.

gsalamin commented 4 years ago

I also get the same error when using variable in the provider configuration with the following version:

Terraform v0.12.26
terraform-provider-kubernetes-alpha v0.1.0

fredo838 commented 4 years ago

I tested https://github.com/hashicorp/terraform-provider-kubernetes-alpha/pull/92, which fixes this issue. This can be closed when the PR gets merged.

manobi commented 4 years ago

I have the exact same problem, I've been using the hardcoded approach (pointing to my kubeconfig) but I have to create a new cluster.

provider "kubernetes-alpha" {
  host                   = local.kube_connection.host
  client_certificate     = local.kube_connection.client_certificate
  client_key             = local.kube_connection.client_key
  cluster_ca_certificate = local.kube_connection.cluster_ca_certificate
  token                  = local.kube_connection.token
}

Produces the following error:

Error: rpc error: code = Unavailable desc = transport is closing

I can also confirm that #92 fixes the issue.

sknight80 commented 4 years ago

Hi,

Do we have any ETAs for the fix to release?

ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!

hashicorp / terraform-provider-kubernetes-alpha