Open erniebilling opened 1 year ago
was able to reproduce this behavior as well
Initial Failure databricks_catalog.catalog: Creating... databricks_storage_credential.cred: Creating... databricks_storage_credential.cred: Creation complete after 1s [id=test-db] databricks_external_location.location: Creating... databricks_external_location.location: Creation complete after 1s [id=test-db] ╷ │ Error: cannot create catalog: User does not have CREATE CATALOG on Metastore 'test-us-west-2'. │ │ with databricks_catalog.catalog, │ on main.tf line 546, in resource "databricks_catalog" "catalog": │ 546: resource "databricks_catalog" "catalog" {
Apply again after a few seconds
databricks_catalog.catalog: Creating... databricks_catalog.catalog: Creation complete after 2s [id=test-cat] databricks_schema.schema["gold"]: Creating... databricks_schema.schema["silver"]: Creating... databricks_schema.schema["bronze"]: Creating... databricks_schema.schema["bronze"]: Creation complete after 0s [id=test-cat.bronze] databricks_schema.schema["silver"]: Creation complete after 0s [id=test-cat.silver] databricks_schema.schema["gold"]: Creation complete after 0s [id=test-cat.gold]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Removing metastore owner (a group is created and assigned as owner) fixes the problem. Instead we allow the principal running terraform to be owner and the group just gets full permissions on the metastore.
Configuration
Expected Behavior
terraform apply
succeeds, workspace, metastore and external tables successfully createdActual Behavior
Table creation often fails with permissions failures like
Steps to Reproduce
In existing Databricks account:
configure DATABRICKS_USERNAME and DATABRICKS_PASSWORD for account with admin rights. configure AWS account credentials (AWS_ACCESS_KEY, AWS_SECRET_ACCESS_KEY, etc)
enter values for
aws_region
,databricks_account_id
,credentials_id
,vpc_name
andsubnet_names
Terraform and provider versions
Terraform v1.3.9 on linux_amd64
Debug Output
tf-debug.log
Important Factoids
Seems to be timing related - retrying the plan/apply eventually works. Really need this to work the first time.