korenlev commented 2 years ago

error from 0.1.6 :


\"kubeconfig-user-5zvlgzwq8s:sdwgk7n6bdhd5gxs6jfk4lf7t942kcdssgb9mgtvmm9q5n9s6gmhlz\"\n\n\ncontexts:\n-
name: \"development\"\n  context:\n    user: \"development\"\n    cluster:
\"development\"\n- name: \"development-masters-2\"\n  context:\n    user:
\"development\"\n    cluster: \"development-masters-2\"\n- name:
\"development-masters-1\"\n  context:\n    user: \"development\"\n    cluster:
\"development-masters-1\"\n- name: \"development-masters-3\"\n  context:\n
user: \"development\"\n    cluster:
\"development-masters-3\"\n\ncurrent-context: \"development\"\n"),
"kube_config_hash":cty.UnknownVal(cty.String)}) does not correlate with any
element in actual.

This is a bug in the provider, which should be reported in the provider's own issue tracker.

xuzhang3 commented 2 years ago

@korenlev Can you share your TF script and the reproduce steps, thanks

korenlev commented 2 years ago

Create new managed k8s cluster using rancher 2.6.2 :

resource "rancher2_cluster" "rancher_cluster" {
  name = var.rancher_managed_cluster_name
  windows_prefered_cluster = (var.windows == "yes" ? true : false)
  cluster_auth_endpoint {
    enabled = true # Allow communicate with the cluster without main rancher management
  }
  rke_config {
   etc .........

TFS service connection creation with kubeconfig :

resource "azuredevops_serviceendpoint_kubernetes" "service_connection" {
  project_id            = var.Project_ID
  service_endpoint_name = "rancher-azure-${var.rancher_managed_cluster_name}"
  apiserver_url         = "https://${var.rancher_url}"
  authorization_type    = "Kubeconfig"

  kubeconfig {
    kube_config            = rancher2_cluster.rancher_cluster.kube_config
    accept_untrusted_certs = true
  }
}

resource "azuredevops_resource_authorization" "auth" {
  project_id  = var.Project_ID
  resource_id = azuredevops_serviceendpoint_kubernetes.service_connection.id
  authorized  = true
}

error (after dump of the actual kubeconfig text

"kube_config_hash":cty.UnknownVal(cty.String)}) does not correlate with any element in actual.

This is a bug in the provider, which should be reported in the provider's own issue tracker.

[error]Bash exited with code '1'.

Finishing: Terraform_run

korenlev commented 2 years ago

masked kcfg string:


2021-12-15T15:31:37.5830571Z module.managed_cluster.azuredevops_serviceendpoint_kubernetes.service_connection 
2021-12-15T15:31:37.5831779Z to include new values learned so far during apply, provider 
2021-12-15T15:31:37.5835148Z "registry.terraform.io/microsoft/azuredevops" produced an invalid new value 
2021-12-15T15:31:37.5836482Z for .kubeconfig: planned set element 
2021-12-15T15:31:37.5837864Z cty.ObjectVal(map[string]cty.Value{"accept_untrusted_certs":cty.True, 
2021-12-15T15:31:37.5839329Z "cluster_context":cty.StringVal(""), "kube_config":cty.StringVal("apiVersion: 
2021-12-15T15:31:37.5841171Z v1\nkind: Config\nclusters:\n- name: \"prod-tools\"\n cluster:\n server: 
2021-12-15T15:31:37.5842808Z \"https://removed.com/k8s/clusters/c-wrwdh\"\n- name: 
2021-12-15T15:31:37.5844340Z \"prod-tools-masters-2\"\n cluster:\n server: 
2021-12-15T15:31:37.5845954Z \"https://1.1.1.1:6443\"\n certificate-authority-data: 
2021-12-15T15:31:37.5847187Z \"0000000000000000\\\n 
2021-12-15T15:31:37.5869840Z klDQVRFLS0tLS0K\"\n- name: \"prod-tools-masters-3\"\n cluster:\n server: 
2021-12-15T15:31:37.5871865Z \"https://1.1.1.2:6443\"\n certificate-authority-data: 
2021-12-15T15:31:37.5873036Z \"0000000000000000\\\n 
2021-12-15T15:31:37.5918623Z klDQVRFLS0tLS0K\"\n- name: \"prod-tools-masters-1\"\n cluster:\n server: 
2021-12-15T15:31:37.5920515Z \"https://1.1.1.3:6443\"\n certificate-authority-data: 
2021-12-15T15:31:37.5921698Z \"0000000000000000\\\n 
2021-12-15T15:31:37.5956118Z klDQVRFLS0tLS0K\"\n\nusers:\n- name: \"prod-tools\"\n user:\n token: 
2021-12-15T15:31:37.5959002Z \"kubeconfig-user-th8n7ww6j6:removed\"\n\n\ncontexts:\n- 
2021-12-15T15:31:37.5960353Z name: \"prod-tools\"\n context:\n user: \"prod-tools\"\n cluster: 
2021-12-15T15:31:37.5961528Z \"prod-tools\"\n- name: \"prod-tools-masters-2\"\n context:\n user: 
2021-12-15T15:31:37.5962602Z \"prod-tools\"\n cluster: \"prod-tools-masters-2\"\n- name: 
2021-12-15T15:31:37.5963701Z \"prod-tools-masters-3\"\n context:\n user: \"prod-tools\"\n cluster: 
2021-12-15T15:31:37.5964822Z \"prod-tools-masters-3\"\n- name: \"prod-tools-masters-1\"\n context:\n 
2021-12-15T15:31:37.5965754Z user: \"prod-tools\"\n cluster: 
2021-12-15T15:31:37.5966681Z \"prod-tools-masters-1\"\n\ncurrent-context: \"prod-tools\"\n"), 
2021-12-15T15:31:37.5967509Z "kube_config_hash":cty.UnknownVal(cty.String)}) does not correlate with any 
2021-12-15T15:31:37.5968163Z element in actual. 
2021-12-15T15:31:37.5968383Z 
2021-12-15T15:31:37.5969224Z This is a bug in the provider, which should be reported in the provider's own 
2021-12-15T15:31:37.5969873Z issue tracker. 
2021-12-15T15:31:37.5970732Z �[0m�[0m 
2021-12-15T15:31:37.8959238Z ##[error]Bash exited with code '1'. 
2021-12-15T15:31:37.8993686Z ##[section]Finishing: Terraform_run

xuzhang3 commented 2 years ago

@korenlev I cannot reproduce your.
kube_config_hash":cty.UnknownVal(cty.String)}) does not correlate with any is not a bug log. From the log, I cannot find any provider crash log. When is the provider crashed, can you paste the errors here? There should be text like nil pointer, invalid address, error ... etc.

korenlev commented 2 years ago

adding again here , it constantly fails to add a rancher 2.6.2 cluster kubeconfig ( just create a cluster and try, was working before that version) : terraform resources:

resource "azuredevops_serviceendpoint_kubernetes" "service_connection" {
  project_id            = var.Project_ID
  service_endpoint_name = var.rancher_managed_cluster_name
  apiserver_url         = "https://${var.rancher_url}"
  authorization_type    = "Kubeconfig"

  kubeconfig {
    kube_config            = rancher2_cluster.rancher_cluster.kube_config
    accept_untrusted_certs = true
  }
}

resource "azuredevops_resource_authorization" "auth" {
  project_id  = var.Project_ID
  resource_id = azuredevops_serviceendpoint_kubernetes.service_connection.id
  authorized  = true
}

================== error log

╷
│ Error: Plugin did not respond
│ 
│   with module.managed_cluster.azuredevops_serviceendpoint_kubernetes.service_connection,
│   on managed/managed.tf line 152, in resource "azuredevops_serviceendpoint_kubernetes" "service_connection":
│  152: resource "azuredevops_serviceendpoint_kubernetes" "service_connection" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
│ contain more details.
╵

Stack trace from the terraform-provider-azuredevops_v0.1.8 plugin:

panic: interface conversion: interface {} is nil, not []interface {}

goroutine 57 [running]:
github.com/microsoft/terraform-provider-azuredevops/azuredevops/internal/service/serviceendpoint.expandServiceEndpointKubernetes(0xc000461e30, 0xc0006aac90, 0xc000195ba0, 0xc000195c70, 0x0)
    github.com/microsoft/terraform-provider-azuredevops/azuredevops/internal/service/serviceendpoint/resource_serviceendpoint_kubernetes.go:211 +0x197f
github.com/microsoft/terraform-provider-azuredevops/azuredevops/internal/service/serviceendpoint.genServiceEndpointCreateFunc.func1(0xc000461e30, 0xfdf180, 0xc00041c200, 0x2, 0x1a0d600)
    github.com/microsoft/terraform-provider-azuredevops/azuredevops/internal/service/serviceendpoint/commons.go:147 +0x65
github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Resource).Apply(0xc000334960, 0xc0001a8eb0, 0xc000791280, 0xfdf180, 0xc00041c200, 0x106c601, 0xc000195bf8, 0xc0006aac90)
    github.com/hashicorp/terraform-plugin-sdk@v1.17.2/helper/schema/resource.go:320 +0x375
github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Provider).Apply(0xc00004ad80, 0xc0003d9a38, 0xc0001a8eb0, 0xc000791280, 0xc0006a9168, 0xc00065b860, 0x106eca0)
    github.com/hashicorp/terraform-plugin-sdk@v1.17.2/helper/schema/provider.go:294 +0x99
github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).ApplyResourceChange(0xc00011cc18, 0x13d6270, 0xc0006686f0, 0xc000461650, 0xc00011cc18, 0xc0006686f0, 0xc0003abba0)
    github.com/hashicorp/terraform-plugin-sdk@v1.17.2/internal/helper/plugin/grpc_provider.go:895 +0x8a5
github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ApplyResourceChange_Handler(0x118dc20, 0xc00011cc18, 0x13d6270, 0xc0006686f0, 0xc000115440, 0x0, 0x13d6270, 0xc0006686f0, 0xc00024e800, 0x38c)
    github.com/hashicorp/terraform-plugin-sdk@v1.17.2/internal/tfplugin5/tfplugin5.pb.go:3305 +0x214
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0003576c0, 0x13e4018, 0xc000703800, 0xc00062a100, 0xc00036eae0, 0x19cb4a0, 0x0, 0x0, 0x0)
    google.golang.org/grpc@v1.32.0/server.go:1194 +0x52b
google.golang.org/grpc.(*Server).handleStream(0xc0003576c0, 0x13e4018, 0xc000703800, 0xc00062a100, 0x0)
    google.golang.org/grpc@v1.32.0/server.go:1517 +0xd0c
google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc000042380, 0xc0003576c0, 0x13e4018, 0xc000703800, 0xc00062a100)
    google.golang.org/grpc@v1.32.0/server.go:859 +0xab
created by google.golang.org/grpc.(*Server).serveStreams.func1
    google.golang.org/grpc@v1.32.0/server.go:857 +0x1fd

Error: The terraform-provider-azuredevops_v0.1.8 plugin crashed!

can be fixed. The output above should help diagnose the issue.

##[error]Bash exited with code '1'.
Finishing: Terraform_run

xuzhang3 commented 2 years ago

korenlev Can you help check the kube_config, looks like you config miss the contexts block.

Example:

contexts:
- context:
  cluster: development
  namespace: frontend
  user: developer
name: dev-frontend

microsoft / terraform-provider-azuredevops

"kube_config_hash":cty.UnknownVal(cty.String)}) does not correlate with any element in actual #500

Create new managed k8s cluster using rancher 2.6.2 :

[error]Bash exited with code '1'.