search

databricks / terraform-provider-databricks

Databricks Terraform Provider
https://registry.terraform.io/providers/databricks/databricks/latest
Other
454 stars 391 forks source link

[ISSUE] Terraform provider keeps re-creating GCP workspace #1807

Closed cfantencent closed 1 year ago

cfantencent commented 1 year ago

Configuration

main.tf

provider "databricks" {
  alias                  = "accounts"
  host                   = "https://accounts.gcp.databricks.com"
  google_service_account = var.databricks_google_service_account
}

resource "databricks_mws_workspaces" "test" {
  provider       = databricks.accounts
  account_id     = var.databricks_account_id
  workspace_name = var.workspace_name
  location       = var.location
  cloud_resource_bucket {
    gcp {
      project_id = var.project
    }
  }
}

provider "databricks" {
  alias                  = "workspace"
  host                   = databricks_mws_workspaces.test.workspace_url
  google_service_account = var.databricks_google_service_account
}

data "databricks_group" "admins" {
  depends_on   = [databricks_mws_workspaces.test]
  provider     = databricks.workspace
  display_name = "admins"
}

resource "databricks_user" "test_adminlist" {
  depends_on = [databricks_mws_workspaces.test]
  provider   = databricks.workspace
  for_each   = toset(var.admin_email_list)
  user_name  = each.value
}

resource "databricks_group_member" "admin_user" {
  for_each   = toset(var.admin_email_list)
  depends_on = [databricks_mws_workspaces.test]
  provider   = databricks.workspace
  group_id   = data.databricks_group.admins.id
  member_id  = databricks_user.test_adminlist[each.value].id
}

resource "databricks_user" "test_userlist" {
  depends_on = [databricks_mws_workspaces.test]
  provider  = databricks.workspace
  for_each  = toset(var.user_email_list)
  user_name = each.value
}

Copy-paste your Terraform configuration here

providers.tf

terraform {
  required_providers {
    databricks = {
      source = "databricks/databricks"
    }
  }
}

provider "google" {
  project = var.project
}

Expected Behavior

Terraform should show no chnage, no add and no destroy

Actual Behavior

╷
│ Error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details
│
│   with databricks_user.test_adminlist["xxx@tencent.com"],
│   on main.tf line 31, in resource "databricks_user" "test_adminlist":
│   31: resource "databricks_user" "test_adminlist" {
│
╵
╷
│ Error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details
│
│   with databricks_user.test_adminlist["xxx@tencent.com"],
│   on main.tf line 31, in resource "databricks_user" "test_adminlist":
│   31: resource "databricks_user" "test_adminlist" {
│
╵
╷
│ Error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details
│
│   with databricks_user.test_userlist["xxx@tencent.com"],
│   on main.tf line 46, in resource "databricks_user" "test_userlist":
│   46: resource "databricks_user" "test_userlist" {
│
╵
╷
│ Error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details
│
│   with databricks_user.test_userlist["xxx@tencent.com"],
│   on main.tf line 46, in resource "databricks_user" "test_userlist":
│   46: resource "databricks_user" "test_userlist" {
│

Steps to Reproduce

`terrafrom plan` first create the resource then the if I run terraform plan again. There will be an auth issue. ``` ### Terraform and provider versions

Debug Output

Important Factoids

nkvuong commented 1 year ago

I suspect this is because you are not using service account impersonation, and a quick glance at the codebase, only impersonation is supported

Let me check with some other folks to understand if this is correct

cfantencent commented 1 year ago

I suspect this is because you are not using service account impersonation, and a quick glance at the codebase, only impersonation is supported

Let me check with some other folks to understand if this is correct Sorry, I am not sure the meaning of service account impersonation. I have exported the following variable "GOOGLE_APPLICATION_CREDENTIALS", "GOOGLE_CREDENTIALS" to make this working in the first apply stage. And I need to use two provider alias since the variable host is different in workspace creatation and user management.

nkvuong commented 1 year ago

This page talks more about the service account impersonation, but essentially:

Creating a workspace requires Databricks account API, whereas creating resources in the workspace requires Databricks workspace API, and these are handled slightly differently in the codebase of the provider.

cfantencent commented 1 year ago

This page talks more about the service account impersonation, but essentially:

  • Either you authenticate directly as the service account by providing its keys
  • Or you grant permission to a user/service account ability to create a short-lived token for said service account and use that to authenticate (called service account impersonation)

Creating a workspace requires Databricks account API, whereas creating resources in the workspace requires Databricks workspace API, and these are handled slightly differently in the codebase of the provider.

Thank you for your reply. So I need to service account. First account: Create workspace, Create second service account, and generate token Second account: Do the workspace management. For the variable GOOGLE_APPLICATION_CREDENTIALS is the token for the first service account GOOGLE_CREDENTIALS is the token for the second account Am I right? Since my current setting can create workspace and users management. But fail to manage them afterwards.

nkvuong commented 1 year ago

actually, could you try updating the provider block for the workspace to be

provider "databricks" {
  alias                  = "workspace"
  auth_type        = "google-creds"
  host                   = databricks_mws_workspaces.test.workspace_url
  google_service_account = var.databricks_google_service_account
}

basically this forces the provider to use google credentials to auth instead of using sa impersonation

cfantencent commented 1 year ago

actually, could you try updating the provider block for the workspace to be

provider "databricks" {
  alias                  = "workspace"
  auth_type        = "google-creds"
  host                   = databricks_mws_workspaces.test.workspace_url
  google_service_account = var.databricks_google_service_account
}

basically this forces the provider to use google credentials to auth instead of using sa impersonation

No it is still not working 

Apply complete! Resources: 7 added, 0 changed, 0 destroyed.
cfan ~/tencnet_work/gcp/databricks_workspace_creation  $ terraform plan
databricks_mws_workspaces.test: Refreshing state... [id=]
databricks_user.test_userlist["c"]: Refreshing state... [id=]
databricks_user.test_adminlist[""]: Refreshing state... [id=]
databricks_user.test_userlist[""]: Refreshing state... [id=]
╷
│ Error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details
│
│   with databricks_user.test_adminlist["xxx@tencent.com"],
│   on main.tf line 33, in resource "databricks_user" "test_adminlist":
│   33: resource "databricks_user" "test_adminlist" {
│
╵
╷
│ Error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details
│
│   with databricks_user.test_adminlist["xxx@tencent.com"],
│   on main.tf line 33, in resource "databricks_user" "test_adminlist":
│   33: resource "databricks_user" "test_adminlist" {
│
╵
╷
│ Error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details
│
│   with databricks_user.test_userlist["xxx@tencent.com"],
│   on main.tf line 48, in resource "databricks_user" "test_userlist":
│   48: resource "databricks_user" "test_userlist" {
│
╵
╷
│ Error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details
│
│   with databricks_user.test_userlist["xxx@tencent.com"],
│   on main.tf line 48, in resource "databricks_user" "test_userlist":
│   48: resource "databricks_user" "test_userlist" {
cfantencent commented 1 year ago

Here is my code block 

provider "databricks" {
  alias                  = "accounts"
  auth_type              = "google-creds"
  host                   = "https://accounts.gcp.databricks.com"
  google_service_account = var.databricks_google_service_account
}

resource "databricks_mws_workspaces" "test" {
  provider       = databricks.accounts
  account_id     = var.databricks_account_id
  workspace_name = var.workspace_name
  location       = var.location
  cloud_resource_bucket {
    gcp {
      project_id = var.project
    }
  }
}

provider "databricks" {
  alias                  = "workspace"
  auth_type              = "google-creds"
  host                   = databricks_mws_workspaces.test.workspace_url
  google_service_account = var.databricks_google_service_account
}
nkvuong commented 1 year ago

could you get the debug log as well? TF_LOG=DEBUG terraform apply -no-color

cfantencent commented 1 year ago

TF_LOG=DEBUG terraform apply -no-color

022-12-02T12:42:38.043+0100 [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: tf_resource_type=databricks_user diagnostic_detail= tf_proto_version=5.3 tf_req_id=987f11ce-f9a2-6a4f-bf72-4afc14717785 tf_provider_addr=registry.terraform.io/databricks/databricks tf_rpc=ReadResource @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_severity=ERROR diagnostic_summary="cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details" timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.043+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring databricks-cli auth, because google-creds is preferred: timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.043+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring pat auth, because google-creds is preferred: timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.043+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring basic auth, because google-creds is preferred: timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.043+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring oauth-m2m auth, because google-creds is preferred: timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.043+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-client-secret auth, because google-creds is preferred: timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.043+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-msi auth, because google-creds is preferred: timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.043+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-cli auth, because google-creds is preferred: timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.044+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring google-accounts auth, because google-creds is preferred: timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.044+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring google-workspace auth, because google-creds is preferred: timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.044+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring databricks-cli auth, because google-creds is preferred: timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.044+0100 [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_severity=ERROR diagnostic_summary="cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details" tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=9af0edc0-9e59-f1fd-3b3e-6cd159ec6b16 tf_rpc=ReadResource @module=sdk.proto diagnostic_detail= tf_proto_version=5.3 tf_resource_type=databricks_user timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.044+0100 [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: tf_provider_addr=registry.terraform.io/databricks/databricks tf_rpc=ReadResource @module=sdk.proto diagnostic_detail = tf_proto_version=5.3 tf_resource_type=databricks_user @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_severity=ERROR diagnostic_summary="cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details" tf_req_id=943e1ae5-5ebd-785d-7700-6a470be4aec1 timestamp=2022-12-02T12:42:38.043+0100 2022-12-02T12:42:38.044+0100 [ERROR] vertex "databricks_user.test_adminlist[\"xxx@tencent.com\"]" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-02T12:42:38.044+0100 [ERROR] vertex "databricks_user.test_adminlist[\"xxx@tencent.com\"]" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-02T12:42:38.044+0100 [ERROR] vertex "databricks_user.test_userlist[\"xxx@tencent.com\"]" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-02T12:42:38.044+0100 [ERROR] vertex "databricks_user.test_userlist[\"xxx@tencent.com\"]" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-02T12:42:38.044+0100 [ERROR] vertex "databricks_user.test_adminlist (expand)" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-02T12:42:38.044+0100 [ERROR] vertex "databricks_user.test_adminlist (expand)" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-02T12:42:38.044+0100 [ERROR] vertex "databricks_user.test_userlist (expand)" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-02T12:42:38.044+0100 [ERROR] vertex "databricks_user.test_userlist (expand)" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-02T12:42:38.044+0100 [INFO] backend/local: plan operation completed

Error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

with databricks_user.test_adminlist["xxx@tencent.com"], on main.tf line 33, in resource "databricks_user" "test_adminlist": 33: resource "databricks_user" "test_adminlist" {

Error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

with databricks_user.test_adminlist["xxx@tencent.com"], on main.tf line 33, in resource "databricks_user" "test_adminlist": 33: resource "databricks_user" "test_adminlist" {

Error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

with databricks_user.test_userlist["xxx@tencent.com"], on main.tf line 48, in resource "databricks_user" "test_userlist": 48: resource "databricks_user" "test_userlist" {

Error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

with databricks_user.test_userlist["xxx@tencent.com"], on main.tf line 48, in resource "databricks_user" "test_userlist": 48: resource "databricks_user" "test_userlist" {

2022-12-02T12:42:38.048+0100 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF" 2022-12-02T12:42:38.049+0100 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.6.5/darwin_amd64/terraform-provider-databricks_v1.6.5 pid=11156 2022-12-02T12:42:38.049+0100 [DEBUG] provider: plugin exited

nkvuong commented 1 year ago

So it looks like google-cred does not work for workspace authentication.

Could you validate if authenticating using service account impersonation works, i.e. keep the original Terraform config, but authenticate with gcloud under a user account/different service account sing gcloud auth application-default login. Just make sure the other identity has Service Account Token creation permission on the service account you've added to Databricks

cfantencent commented 1 year ago

So it looks like google-cred does not work for workspace authentication.

Could you validate if authenticating using service account impersonation works, i.e. keep the original Terraform config, but authenticate with gcloud under a user account/different service account sing gcloud auth application-default login. Just make sure the other identity has Service Account Token creation permission on the service account you've added to Databricks

Yes, the service account has the Service Account Token creation permission. But I do not get the gcloud part. Is it really necessary to use gcloud login to perform the change? Can I just provide the service account and token to make it work?

nkvuong commented 1 year ago

The gcloud login is for user account authentication, if you're using a service account then I don't think it's necessary. Could I just double check the set up:

Another thing to check is whether the value of databricks_mws_workspaces.test.workspace_url is populated correctly. Easiest way to test is to replace host in the workspace provider with the url string of the workspace

host                   = databricks_mws_workspaces.test.workspace_url
cfantencent commented 1 year ago

The gcloud login is for user account authentication, if you're using a service account then I don't think it's necessary. Could I just double check the set up:

  • Service account A has the token set as env variable, and permission to create token for service account B
  • Service account B is provided to Terraform, and has been added to the Databricks account

Another thing to check is whether the value of databricks_mws_workspaces.test.workspace_url is populated correctly. Easiest way to test is to replace host in the workspace provider with the url string of the workspace

host                   = databricks_mws_workspaces.test.workspace_url

Thank you for your reply. So here goes the service account B, authed with variable GOOGLE_APPLICATION_CREDENTIALS

provider "databricks" {
alias                  = "accounts"
host                   = "https://accounts.gcp.databricks.com"
google_service_account = var.databricks_google_service_account
}

And here goes service account A, authed with variable GOOGLE_CREDENTIALS

provider "databricks" {
alias                  = "workspace"
host                   = databricks_mws_workspaces.test.workspace_url
google_service_account = var.databricks_google_service_account
}

And databricks_mws_workspaces.test.workspace_url is populated correctly.

nkvuong commented 1 year ago

Oh, it should be simpler than that - you specify the same service account B for both provider blocks (one at account and one at workspace), and then auth using service account A as GOOGLE_APPLICATION_CREDENTIALS. The provider will automatically handle the impersonation

cfantencent commented 1 year ago

TF_LOG=DEBUG terraform apply -no-color

After I changed service account setting the error seems the same. 2022-12-05T12:48:43.929+0100 [DEBUG] ReferenceTransformer: "databricks_user.test_adminlist[\"xxx@tencent.com\"]" references: [] 2022-12-05T12:48:43.931+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring pat auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.931+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring basic auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.931+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring oauth-m2m auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.931+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-client-secret auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.931+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-msi auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.931+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-cli auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.931+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring google-accounts auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.931+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring google-workspace auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.931+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring databricks-cli auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring pat auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring basic auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring oauth-m2m auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-client-secret auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-msi auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.931+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-cli auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring google-accounts auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring google-workspace auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring databricks-cli auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: tf_rpc=ReadResource @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_severity=ERROR tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/databricks/databricks @module=sdk.proto diagnostic_detail= diagnostic_summary="cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details" tf_req_id=a5052107-fc03-336f-e340-c42dc3c086df tf_resource_type=databricks_user timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring pat auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring basic auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring oauth-m2m auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-client-secret auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-msi auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-cli auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring google-accounts auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring google-workspace auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring databricks-cli auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: @module=sdk.proto diagnostic_detail= diagnostic_severity=ERROR tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/databricks/databricks @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 tf_resource_type=databricks_user diagnostic_summary="cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details" tf_req_id=552194a7-b9f5-365c-d7c9-e97c3531f6aa tf_rpc=ReadResource timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring pat auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring basic auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring oauth-m2m auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-client-secret auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-msi auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring azure-cli auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring google-accounts auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: tf_proto_version=5.3 tf_req_id=23c5db50-ca1e-5e84-3593-e03f01b76d7a @module=sdk.proto diagnostic_detail= diagnostic_summary="cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details" tf_provider_addr=registry.terraform.io/databricks/databricks tf_resource_type=databricks_user tf_rpc=ReadResource @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_severity=ERROR timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring google-workspace auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [INFO] provider.terraform-provider-databricks_v1.6.5: Ignoring databricks-cli auth, because google-creds is preferred: timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: diagnostic_severity=ERROR tf_proto_version=5.3 @module=sdk.proto diagnostic_detail= diagnostic_summary="cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details" tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=ca4561fa-1269-ddc4-c2c0-2811adcfbe19 tf_resource_type=databricks_user @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 tf_rpc=ReadResource timestamp=2022-12-05T12:48:43.932+0100 2022-12-05T12:48:43.932+0100 [ERROR] vertex "databricks_user.test_userlist[\"xxx@tencent.com\"]" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T12:48:43.932+0100 [ERROR] vertex "databricks_user.test_adminlist[\"xxx@tencent.com\"]" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T12:48:43.932+0100 [ERROR] vertex "databricks_user.test_userlist[\"xxx@tencent.com\"]" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T12:48:43.932+0100 [ERROR] vertex "databricks_user.test_adminlist[\"xxx@tencent.com\"]" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T12:48:43.933+0100 [ERROR] vertex "databricks_user.test_userlist (expand)" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T12:48:43.933+0100 [ERROR] vertex "databricks_user.test_userlist (expand)" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T12:48:43.933+0100 [ERROR] vertex "databricks_user.test_adminlist (expand)" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T12:48:43.933+0100 [ERROR] vertex "databricks_user.test_adminlist (expand)" error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T12:48:43.933+0100 [INFO] backend/local: plan operation completed

Error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

with databricks_user.test_adminlist["xxx@tencent.com"], on main.tf line 33, in resource "databricks_user" "test_adminlist": 33: resource "databricks_user" "test_adminlist" {

Error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

with databricks_user.test_adminlist["xxx@tencent.com"], on main.tf line 33, in resource "databricks_user" "test_adminlist": 33: resource "databricks_user" "test_adminlist" {

Error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

with databricks_user.test_userlist["xxx@tencent.com"], on main.tf line 48, in resource "databricks_user" "test_userlist": 48: resource "databricks_user" "test_userlist" {

Error: cannot read user: cannot configure google-creds auth. Attributes used: auth_type, google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

with databricks_user.test_userlist["xxx@tencent.com"], on main.tf line 48, in resource "databricks_user" "test_userlist": 48: resource "databricks_user" "test_userlist" {

2022-12-05T12:48:43.936+0100 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF" 2022-12-05T12:48:43.937+0100 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.6.5/darwin_amd64/terraform-provider-databricks_v1.6.5 pid=73707 2022-12-05T12:48:43.937+0100 [DEBUG] provider: plugin exited

cfantencent commented 1 year ago

Account A has the permissions: Editor ,Service Account Token Creator
Account B has the permissions: Editor, Role Administrator and Service Account Token Creator

nkvuong commented 1 year ago

Could you remove the auth_type = "google-creds" and see if the error persists

cfantencent commented 1 year ago

Could you remove the auth_type = "google-creds" and see if the error persists

Sorry still not working

cfantencent commented 1 year ago

2022-12-05T13:01:12.960+0100 [DEBUG] ReferenceTransformer: "databricks_user.test_adminlist[\"xxx@tencent.com\"]" references: [] 2022-12-05T13:01:12.963+0100 [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: diagnostic_detail= diagnostic_severity=ERROR tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=d7221002-999c-2433-30cb-e4adb44aae84 tf_resource_type=databricks_user tf_rpc=ReadResource @module=sdk.proto diagnostic_summary="cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details" tf_proto_version=5.3 @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 timestamp=2022-12-05T13:01:12.963+0100 2022-12-05T13:01:12.963+0100 [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: diagnostic_severity=ERROR tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=0b3b98c0-45e4-d87b-a08e-b48751e3eafc tf_proto_version=5.3 @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_detail= diagnostic_summary="cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details" tf_resource_type=databricks_user tf_rpc=ReadResource timestamp=2022-12-05T13:01:12.963+0100 2022-12-05T13:01:12.963+0100 [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: @module=sdk.proto tf_provider_addr=registry.terraform.io/databricks/databricks @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_detail= diagnostic_severity=ERROR diagnostic_summary="cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details" tf_proto_version=5.3 tf_req_id=77c8b68a-771c-ebad-a47b-872976018116 tf_resource_type=databricks_user tf_rpc=ReadResource timestamp=2022-12-05T13:01:12.963+0100 2022-12-05T13:01:12.963+0100 [ERROR] vertex "databricks_user.test_userlist[\"xxx@tencent.com\"]" error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T13:01:12.963+0100 [ERROR] vertex "databricks_user.test_adminlist[\"xxx@tencent.com\"]" error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T13:01:12.963+0100 [ERROR] vertex "databricks_user.test_userlist[\"xxx@tencent.com\"]" error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T13:01:12.963+0100 [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: diagnostic_detail= diagnostic_severity=ERROR @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_summary="cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details" tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=ee2c3352-6ba8-8d15-88e9-7cf85c1e2bcc tf_resource_type=databricks_user tf_rpc=ReadResource timestamp=2022-12-05T13:01:12.963+0100 2022-12-05T13:01:12.963+0100 [ERROR] vertex "databricks_user.test_userlist (expand)" error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T13:01:12.963+0100 [ERROR] vertex "databricks_user.test_userlist (expand)" error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T13:01:12.963+0100 [ERROR] vertex "databricks_user.test_adminlist[\"xxxx@tencent.com\"]" error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T13:01:12.963+0100 [ERROR] vertex "databricks_user.test_adminlist (expand)" error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T13:01:12.963+0100 [ERROR] vertex "databricks_user.test_adminlist (expand)" error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details 2022-12-05T13:01:12.963+0100 [INFO] backend/local: plan operation completed

Error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

with databricks_user.test_adminlist["xxx@tencent.com"], on main.tf line 31, in resource "databricks_user" "test_adminlist": 31: resource "databricks_user" "test_adminlist" {

Error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

with databricks_user.test_adminlist["xxx@tencent.com"], on main.tf line 31, in resource "databricks_user" "test_adminlist": 31: resource "databricks_user" "test_adminlist" {

Error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

with databricks_user.test_userlist["xxx@tencent.com"], on main.tf line 46, in resource "databricks_user" "test_userlist": 46: resource "databricks_user" "test_userlist" {

Error: cannot read user: cannot configure google-workspace auth: could not obtain OIDC token. impersonate: an audience must be provided Running 'gcloud auth application-default login' may help. Attributes used: google_service_account. Environment variables used: GOOGLE_CREDENTIALS. Please check https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication for details

with databricks_user.test_userlist["xxx@tencent.com"], on main.tf line 46, in resource "databricks_user" "test_userlist": 46: resource "databricks_user" "test_userlist" {

2022-12-05T13:01:12.967+0100 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF" 2022-12-05T13:01:12.968+0100 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.6.5/darwin_amd64/terraform-provider-databricks_v1.6.5 pid=77255 2022-12-05T13:01:12.968+0100 [DEBUG] provider: plugin exited

cfantencent commented 1 year ago

For me the wired part is that terraform apply is successful and all the resource has been added. But terraform plan is not working after. So the same auth method was working first time but after not.

microcassidy commented 1 year ago

I am getting the same error. My initial apply is successful and then all subsequent operations fail.

cfantencent commented 1 year ago

I am getting the same error. My initial apply is successful and then all subsequent operations fail.

You can separate workspace creation and workspace management as a work around.

microcassidy commented 1 year ago

Hardcoding the host in the workspace provider block allows terraform plan to be ran. It seems like the host isn't available after the initial apply perhaps? It seems weird that we are able to create and manage a workspace once and once only...

@cfantencent could you elaborate please?

cfantencent commented 1 year ago

Hardcoding the host in the workspace provider block allows terraform plan to be ran. It seems like the host isn't available after the initial apply perhaps? It seems weird that we are able to create and manage a workspace once and once only...

@cfantencent could you elaborate please? Yes, I got the same issue. I can not manage workspace by terraform. Once workspace is created, no matter there is any change in config or not, the workspace will be recreated. And cluster management needs the dependency of the creatation of workspace. So basically, I am stucked on using terraform manage databricks.

nkvuong commented 1 year ago

@cfantencent @xuky76

Once workspace is created, no matter there is any change in config or not, the workspace will be recreated

I think this is the root cause - because Terraform wants to re-create the workspace, the databricks_mws_workspaces.test.workspace_url will change. This means the provider using that output as a host won't be configured correctly and failed.

The plan fails as Terraform will read all data sources, and the provider is not configured correctly.

What attribute is changing resulting in the provider wanting to re-create the workspace?

cfantencent commented 1 year ago

databricks_mws_workspaces.test: Refreshing state... [id=5fe34970-bdbb-4353-bbfd-dabecca821e3/3769966105657340]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: -/+ destroy and then create replacement

Terraform will perform the following actions:

databricks_mws_workspaces.test must be replaced

-/+ resource "databricks_mws_workspaces" "test" { ~ cloud = "gcp" -> (known after apply) ~ creation_time = 1670948473232 -> (known after apply)

Plan: 1 to add, 0 to change, 1 to destroy.

nkvuong commented 1 year ago

so the bug is that the network information being returned, and this is causing the provider to attempt to recreate the workspace

Cease2A commented 1 year ago

databricks_mws_workspaces.test: Refreshing state... [id=5fe34970-bdbb-4353-bbfd-dabecca821e3/3769966105657340]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: -/+ destroy and then create replacement

Terraform will perform the following actions:

databricks_mws_workspaces.test must be replaced

-/+ resource "databricks_mws_workspaces" "test" { ~ cloud = "gcp" -> (known after apply) ~ creation_time = 1670948473232 -> (known after apply) - deployment_name = "3769966105657340.0" -> null ~ id = "5fe34970-bdbb-4353-bbfd-dabecca821e3/3769966105657340" -> (known after apply) ~ pricing_tier = "PREMIUM" -> (known after apply) ~ workspace_id = 3769966105657340 -> (known after apply) ~ workspace_status = "RUNNING" -> (known after apply) ~ workspace_status_message = "Workspace is running." -> (known after apply) ~ workspace_url = "https://3769966105657340.0.gcp.databricks.com" -> (known after apply) # (4 unchanged attributes hidden)

  - network { # forces replacement
      - gcp_common_network_config {
          - gke_cluster_master_ip_range = "10.3.0.0/28" -> null
          - gke_connectivity_type       = "PRIVATE_NODE_PUBLIC_MASTER" -> null
        }

      - gcp_managed_network_config {
          - gke_cluster_pod_ip_range     = "10.1.0.0/16" -> null
          - gke_cluster_service_ip_range = "10.2.0.0/20" -> null
          - subnet_cidr                  = "10.0.0.0/16" -> null
        }
    }

    # (1 unchanged block hidden)
}

Plan: 1 to add, 0 to change, 1 to destroy.

Hardcoding the host in the workspace provider block allows terraform plan to be ran. It seems like the host isn't available after the initial apply perhaps? It seems weird that we are able to create and manage a workspace once and once only... @cfantencent could you elaborate please? Yes, I got the same issue. I can not manage workspace by terraform. Once workspace is created, no matter there is any change in config or not, the workspace will be recreated. And cluster management needs the dependency of the creatation of workspace. So basically, I am stucked on using terraform manage databricks.

@cfantencent can you please let me know whether the fix is working for you or not, I'm facing the same issue.