databricks / terraform-provider-databricks

Databricks Terraform Provider
https://registry.terraform.io/providers/databricks/databricks/latest
Other
424 stars 366 forks source link

[ISSUE] Databricks AWS - databricks_mws_storage_configurations and databricks_storage_credential unable to key in IAM role ARN #3639

Open habhabhabs opened 1 month ago

habhabhabs commented 1 month ago

Configuration

https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/storage_credential https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/mws_workspaces

The Terraform databricks_metastore and databricks_mws_storage_configurations resource block unable to provide IAM role ARN for S3 access. This had led to issues with mounting the DBFS for storage purposes, as well as pertinent caching issues with the metastore due to the lack of IAM role ARN provided during configuration of the storage configuration.

databricks_metastore missing IAM role ARN field in Terraform - however available in UI

Screenshot 2024-06-04 at 8 08 17 AM

databricks_mws_storage_configurations missing IAM role ARN field in Terraform - however available in UI

Screenshot 2024-06-03 at 7 30 33 PM
 resource "databricks_metastore" "metastore" {
   provider      = databricks.mws
   name          = "xxxx-unity-catalog-metastore"
   region        = "ap-southeast-1"
   force_destroy = true
   storage_root  = "s3://xxx/metastore"
}

resource "databricks_mws_storage_configurations" "this" {
  provider                   = databricks.mws
  account_id                 = var.databricks_account_id
  storage_configuration_name = "${var.prefix}-storage"
  bucket_name                = aws_s3_bucket.root_storage_bucket.bucket
}

Expected Behavior

User should be able to enter a IAM role ARN in the databricks_metastore and databricks_mws_storage_configurations Terraform resource blocks.

Actual Behavior

User is not given the option to enter the IAM role ARN in the databricks_metastore and databricks_mws_storage_configurations Terraform resource blocks.

Steps to Reproduce

Terraform v1.46.0 (latest as of writing)

Is it a regression?

No

Debug Output

NA

Important Factoids

NA

Would you like to implement a fix?

KIV

alexott commented 4 weeks ago

Both of them aren't exposed via APIs