[ISSUE] Issue with `databricks_secret_scope` resource

[imathu]

Configuration

resource "databricks_secret_scope" "vault_backed_sc" {
  name     = "test-secret"
  provider = databricks.workspace

  keyvault_metadata {
    resource_id = azurerm_key_vault.main.id
    dns_name    = azurerm_key_vault.main.vault_uri
  }
}

Expected Behavior

"databricks_secret_scope" should only create the secret scope resource. It should not create an additional Key Vault access policy. An access policy is a separate resource in terms of infrastructure-as-code. An access_policy can have another lifecycle than the scope, and it can be managed with a different terraform resource. However, if "databricks_secret_scope" creates the access policy, the policy should be removed, if "databricks_secret_scope" is deleted. If you create more than one scope, the policy should remain until the last scope is deleted. This makes things cumbersome. Therefore, creating the scope and creating the policy should be kept in different resources.

Actual Behavior

"databricks_secret_scope" automatically creates a Vault access policy for the Service Principal "AzureDatabricks". If you delete the databricks_secret_scope, the policy still remains on the Key Vault.

Steps to Reproduce

1. terraform apply
2. Browse your key vault access policy and look for a policy with name "AzureDatabricks"
3. Remove databricks_secret_scope
4. terraform apply
5. Browse your key vault access policy. The policy for "AzureDatabricks" still remains.

Terraform and provider versions

Terraform v1.3.6 on darwin_arm64

• provider registry.terraform.io/hashicorp/azuread v2.47.0
• provider registry.terraform.io/hashicorp/azurerm v3.89.0
• provider registry.terraform.io/databricks/databricks v1.35.0

Is it a regression?

I guess some time ago, you had to explicitly create the vault access policy apart from the secret scope. I don't remember the version back then.

Debug Output

No debugging output as there is no error.

Important Factoids

Would you like to implement a fix?

For backward compatibility, I could implement a flag in the terraform resource as follows:

resource "databricks_secret_scope" "vault_backed_sc" {
  name     = "test-secret"
  auto_create_policy = "false"
  ...
}

This is still a hack, but at least, I could control the policy on my own, uncoupled from the secret scope. Or I could delete the code that is creating the access policy.

[alexott]

Databricks provider doesn't operate on the Azure APIs, so we can't do this. You need to set policies/rbac roles accordingly... I.e., like this: https://github.com/alexott/terraform-playground/blob/main/adb-workspace-cmk-rbac/keyvault.tf#L103