[ISSUE] Issue with `databricks_metastore_assignment` resource

[arothian]

The databricks_metastore_assignment incorrectly refreshes state from the API.

Configuration

resource "databricks_metastore_assignment" "metastore_assignment" {
  provider             = databricks.workspace
  workspace_id         = databricks_mws_workspaces.this.workspace_id
  metastore_id         = databricks_metastore.metastore.id
  default_catalog_name = var.default_catalog_name
}

databricks.workspace points to a provider configuration interacting with the workspace API, using workspace created externally from the TF code. Let's call it Workspace Control databricks_mws_workspaces.this defines a workspace being managed, Workspace This. databricks_metastore.metastore.id defines a metastore being managed.

Expected Behavior

When terraform plan is executed, the state should refresh and correctly mirror the actual resources.

Actual Behavior

When first applied, Workspace This is correctly associated to the metastore. However, subsequent refreshes of state, such as those when executing terraform plan, indicate that the association needs to be updated:

  ~ resource "databricks_metastore_assignment" "metastore_assignment" {
        id                   = "<workspace-this-id>|<metastore.id>"
      ~ metastore_id         = "<metastore for workspace control>" -> "<metastore.id"
        # (2 unchanged attributes hidden)
    }

This seems to occur because of the API call used during read simply returns the metastore of the workspace associated with the databricks provider. If your provider is a different workspace than the one identified in the resource, it retrieves the wrong metastore id.

Steps to Reproduce

terraform plan

Terraform and provider versions

Terraform v1.4.6
on darwin_arm64
+ provider registry.terraform.io/databricks/databricks v1.21.0
+ provider registry.terraform.io/hashicorp/aws v5.7.0

Debug Output

NA

Important Factoids

This occurs because the databricks provider is using a different workspace than the metastore assignment workspace being managed. The metastore for the workspace used by the provider is not the same as the one being associated in the terraform resource. The resource allows you to create these assignments, but then it doesn't correctly read the state afterwards. Moving this resource to use the account-level API would resolve this issue as there is no workspace-level API to read metastore assignments, only create, update and delete.

[arothian]

Similar to #2061 but under different circumstances.

[arothian]

Looks like PR #2182 should resolve this by using the account level API

[mgyucht]

Can you try the account-level API and verify that this issue is resolved?