search

databricks / terraform-provider-databricks

Databricks Terraform Provider
https://registry.terraform.io/providers/databricks/databricks/latest
Other
445 stars 384 forks source link

[ISSUE] Issue with `databricks_metastore_assignment` resource #2379

Closed rishabhtrivedi23 closed 1 year ago

rishabhtrivedi23 commented 1 year ago

Configuration

# Copy-paste your Terraform configuration here
provider.tf

provider "databricks" {
  alias                  = "accounts"
  host                   = var.account_console_url
  account_id             = var.databricks_account_id
  google_service_account = var.google_service_account_email
  rate_limit             = 5
}

provider "databricks" {
  alias                  = "workspace"
  host                   = var.admin_workspace_url
  google_service_account = var.google_service_account_email
  rate_limit             = 5
}

main.tf

resource "databricks_metastore_assignment" "assignment" {
  provider = databricks.ws
  metastore_id = var.metastore_id
  workspace_id = databricks_mws_workspaces.workspace.workspace_id
}

Expected Behavior

It should have performed metastore assignment.

Terraform will perform the following actions:

module.workspace.databricks_metastore_assignment.assignment will be created

Actual Behavior

It fails at terraform level but performs metastore assignment at databricks account console level. module.workspace.databricks_metastore_assignment.assignment: Creating... ╷ │ Error: cannot read metastore assignment: No metastore assigned for the current workspace. │ │ with module.workspace.databricks_metastore_assignment.assignment, │ on ........\modules\databricks\buildworkspace\buildworkspace.tf line 61, in resource "databricks_metastore_assignment" "assignment": │ 61: resource "databricks_metastore_assignment" "assignment" { │

Steps to Reproduce

Terraform and provider versions

terraform version Terraform v1.4.6 on windows_amd64

Debug Output

Important Factoids

I have added debug output below :

module.workspace.databricks_metastore_assignment.assignment: Creating... 2023-06-06T07:57:34.092Z [INFO] Starting apply for module.workspace.databricks_metastore_assignment.assignment 2023-06-06T07:57:34.093Z [DEBUG] module.workspace.databricks_metastore_assignment.assignment: applying the planned Create change 2023-06-06T07:57:34.157Z [INFO] provider.terraform-provider-databricks_v1.17.0.exe: Using Google Default Application Credentials for Workspace: timestamp=2023-06-06T07:57:34.129Z 2023-06-06T07:57:35.222Z [DEBUG] provider.terraform-provider-databricks_v1.17.0.exe: PUT /api/2.1/unity-catalog/workspaces//metastore

{ "default_catalog_name": "hive_metastore", "metastore_id": "", "workspace_id": } < HTTP/2.0 200 OK < {}: timestamp=2023-06-06T07:57:35.222Z 2023-06-06T07:57:36.067Z [DEBUG] provider.terraform-provider-databricks_v1.17.0.exe: GET /api/2.1/unity-catalog/metastore_summary < HTTP/2.0 404 Not Found < { < "details": [ < { < "@type": "type.googleapis.com/google.rpc.RequestInfo", < "request_id": "926b9dfa-8b6f-4131-a2a1-feb572747c18", < "serving_data": "" < } < ], < "error_code": "METASTORE_DOES_NOT_EXIST", < "message": "No metastore assigned for the current workspace." < }: timestamp=2023-06-06T07:57:36.067Z 2023-06-06T07:57:36.074Z [ERROR] provider.terraform-provider-databricks_v1.17.0.exe: Response contains error diagnostic: @module=sdk.proto tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/databricks/databricks tf_rpc=ApplyResourceChange @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_severity=ERROR diagnostic_summary="cannot read metastore assignment: No metastore assigned for the current workspace." tf_req_id=847d1731-9907-55d1-4037-d806cff20e68 tf_resource_type=databricks_metastore_assignment diagnostic_detail= timestamp=2023-06-06T07:57:36.067Z 2023-06-06T07:57:36.077Z [ERROR] vertex "module.workspace.databricks_metastore_assignment.assignment" error: cannot read metastore assignment: No metastore assigned for the current workspace.

Error: cannot read metastore assignment: No metastore assigned for the current workspace.

with module.workspace.databricks_metastore_assignment.assignment, on ........\modules\databricks\buildworkspace\buildworkspace.tf line 61, in resource "databricks_metastore_assignment" "assignment": 61: resource "databricks_metastore_assignment" "assignment" {

2023-06-06T07:57:36.177Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF" 2023-06-06T07:57:36.188Z [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.17.0/windows_amd64/terraform-provider-databricks_v1.17.0.exe pid=7608 2023-06-06T07:57:36.188Z [DEBUG] provider: plugin exited

alexott commented 1 year ago

provider = databricks.ws should be provider = databricks.workspace ?

rishabhtrivedi23 commented 1 year ago

this is what I am defining while calling

module "workspace" { providers = { databricks.ac = databricks.accounts databricks.ws = databricks.workspace }

rishabhtrivedi23 commented 1 year ago

I can also see metastore assignment block in tfstate file as "tainted" and the metastore ID is missing.

  "module": "module.workspace",
  "mode": "managed",
  "type": "databricks_metastore_assignment",
  "name": "assignment",
  "provider": "provider[\"registry.terraform.io/databricks/databricks\"].workspace",
  "instances": [
    {
      "status": "**tainted**",
      "schema_version": 0,
      "attributes": {
        "default_catalog_name": "hive_metastore",
        "id": "<workspace_id>|<metastore_id>",
        **"metastore_id": "",**
        "workspace_id": <workspace_id>
      },
      "sensitive_attributes": [],
      "private": "bnVsbA==",
      "dependencies": [
        "module.workspace.databricks_mws_workspaces.workspace"
      ]
    }
rishabhtrivedi23 commented 1 year ago

I also tried untaint but it didn't work.

rishabhtrivedi23 commented 1 year ago

This is resolved after updating provider.tf and using a workspace which is already added to metastore.