[ISSUE] Issue with `databricks_mws_permission_assignment` resource

KingMichaelPark commented 1 year ago

Configuration

The root file creates the account level groups and users in the root module, the databricks.mws provider is used exclusively there. It also used exclusively in the staging and production workspaces. The staging workspace already exists, so the groups exist and identity federation for that workspace exists.

// initialize provider in "MWS" mode to provision new workspace
provider "databricks" {
  alias      = "mws"
  host       = "https://accounts.cloud.databricks.com"
  account_id = var.databricks_account_id
  username   = var.databricks_account_username
  password   = var.databricks_account_password
}

module "root" {
  source = "./modules/aws_root"

  databricks_account_id = var.databricks_account_id

  tags = var.tags

  existing_admins             = var.existing_admins
  cidr_block                  = var.cidr_block
  databricks_metastore_admins = var.databricks_metastore_admins
  databricks_bronze_users     = var.databricks_bronze_users
  databricks_silver_users     = var.databricks_silver_users
  databricks_gold_users       = var.databricks_gold_users

  unity_admin_group    = var.unity_admin_group
  aws_s3_log_bucket_id = aws_s3_bucket.logging_bucket.id

  providers = {
    databricks.mws = databricks.mws
  }

}

# Workspace Definitions

module "staging" {
  source = "./modules/aws_workspace"

  # existing_admins                                        = var.existing_admins
  databricks_account_id                                  = var.databricks_account_id
  databricks_mws_credentials_credentials_id              = module.root.databricks_mws_credentials_credentials_id
  databricks_mws_networks_network_id                     = module.root.databricks_mws_networks_network_id
  databricks_mws_storage_configurations_configuration_id = module.root.databricks_mws_storage_configurations_configuration_id
  bronze_group_id                                        = module.root.bronze_group_id
  silver_group_id                                        = module.root.silver_group_id
  gold_group_id                                          = module.root.gold_group_id
  metastore_admin_group_id                               = module.root.metastore_admin_group_id
  databricks_service_principal_id                        = module.root.databricks_service_principal_id

  env    = "staging"
  region = var.region

  providers = {
    databricks.mws = databricks.mws
  }
  depends_on = [module.root]
}

module "production" {
  source = "./modules/aws_workspace"

  databricks_account_id                                  = var.databricks_account_id
  databricks_mws_credentials_credentials_id              = module.root.databricks_mws_credentials_credentials_id
  databricks_mws_networks_network_id                     = module.root.databricks_mws_networks_network_id
  databricks_mws_storage_configurations_configuration_id = module.root.databricks_mws_storage_configurations_configuration_id
  bronze_group_id                                        = module.root.bronze_group_id
  silver_group_id                                        = module.root.silver_group_id
  gold_group_id                                          = module.root.gold_group_id
  metastore_admin_group_id                               = module.root.metastore_admin_group_id
  databricks_service_principal_id                        = module.root.databricks_service_principal_id

  env    = "production"
  region = var.region

  providers = {
    databricks.mws = databricks.mws
  }
  depends_on = [module.root]
}

Module root outputs.tf

output "databricks_mws_credentials_credentials_id" {
  value = databricks_mws_credentials.this.credentials_id
}
output "databricks_mws_storage_configurations_configuration_id" {
  value = databricks_mws_storage_configurations.this.storage_configuration_id
}
output "databricks_mws_networks_network_id" {
  value = databricks_mws_networks.this.network_id
}

output "metastore_admin_group_id" {
  value = databricks_group.admins.id
}

output "bronze_group_id" {
  value = databricks_group.bronze.id
}

output "silver_group_id" {
  value = databricks_group.silver.id
}

output "gold_group_id" {
  value = databricks_group.gold.id
}

output "bronze_group_name" {
  value = databricks_group.bronze.display_name
}

output "silver_group_name" {
  value = databricks_group.silver.display_name
}

output "gold_group_name" {
  value = databricks_group.gold.display_name
}

output "databricks_service_principal_application_id" {
  value = databricks_service_principal.sp.application_id
}

output "databricks_service_principal_id" {
  value = databricks_service_principal.sp.id
}

output "cross_account_role_name" {
  value = aws_iam_role.cross_account_role.name
}

This is the file in ./modules/aws_workspace that is referenced above.

resource "databricks_mws_workspaces" "this" {
  provider       = databricks.mws
  account_id     = var.databricks_account_id
  aws_region     = var.region
  workspace_name = var.env

  credentials_id           = var.databricks_mws_credentials_credentials_id
  storage_configuration_id = var.databricks_mws_storage_configurations_configuration_id
  network_id               = var.databricks_mws_networks_network_id

  token {
    comment = "Terraform ${var.env}"
  }
}

resource "databricks_mws_permission_assignment" "add_metastore_admins" {
  provider     = databricks.mws
  workspace_id = databricks_mws_workspaces.this.workspace_id
  principal_id = var.metastore_admin_group_id
  permissions  = ["ADMIN"]
  depends_on = [
    databricks_mws_workspaces.this
  ]
}

resource "databricks_mws_permission_assignment" "bronze" {
  provider     = databricks.mws
  workspace_id = databricks_mws_workspaces.this.workspace_id
  principal_id = var.bronze_group_id
  permissions  = ["USER"]
  depends_on = [
    databricks_mws_workspaces.this
  ]
}

resource "databricks_mws_permission_assignment" "silver" {
  provider     = databricks.mws
  workspace_id = databricks_mws_workspaces.this.workspace_id
  principal_id = var.silver_group_id
  permissions  = ["USER"]
  depends_on = [
    databricks_mws_workspaces.this
  ]
}

resource "databricks_mws_permission_assignment" "gold" {
  provider     = databricks.mws
  workspace_id = databricks_mws_workspaces.this.workspace_id
  principal_id = var.gold_group_id
  permissions  = ["USER"]
  depends_on = [
    databricks_mws_workspaces.this
  ]
}

resource "databricks_mws_permission_assignment" "sp" {
  provider     = databricks.mws
  workspace_id = databricks_mws_workspaces.this.workspace_id
  principal_id = var.databricks_service_principal_id
  permissions  = ["ADMIN"]
  depends_on = [
    databricks_mws_workspaces.this
  ]
}

Expected Behavior

The account level groups (and their users) are added to the workspace like they were when added with just the staging module enabled.

Actual Behavior

Enabling the production module does not allow for the groups to be assigned to the workspace. Failing with the message:

Error: cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/{ACCOUNT_ID}/workspaces/{WORKSPACE_NUMBER/permissionassignments/principals/{PRINCIPAL_ID) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace
│
│   with module.production.databricks_mws_permission_assignment.add_metastore_admins,
│   on modules/aws_workspace/db_workspaces.tf line 28, in resource "databricks_mws_permission_assignment" "add_metastore_admins":
│   28: resource "databricks_mws_permission_assignment" "add_metastore_admins" {

That output comes for each of the databricks_mws_permission_assignment resources for the production module. The staging module already successfully created. It seems to only occur once enabling a second module.

Steps to Reproduce

terraform apply

The plan (only showing the first)


Terraform will perform the following actions:

  # module.production.databricks_mws_permission_assignment.add_metastore_admins will be created
  + resource "databricks_mws_permission_assignment" "add_metastore_admins" {
      + id           = (known after apply)
      + permissions  = [
          + "ADMIN",
        ]
      + principal_id = 82234234234234
      + workspace_id = 83034213312342363
    }

Terraform and provider versions

Debug Output

2022-11-29T11:01:56.277Z [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 tf_resource_type=databricks_mws_permission_assignment diagnostic_summary="cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/1085788622452483) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace" tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/databricks/databricks @module=sdk.proto diagnostic_detail= diagnostic_severity=ERROR tf_req_id=d6317e78-18aa-82eb-9b51-ff6e7dd479b5 tf_rpc=ApplyResourceChange timestamp=2022-11-29T11:01:56.277Z
2022-11-29T11:01:56.281Z [ERROR] vertex "module.production.databricks_mws_permission_assignment.bronze" error: cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/1085788622452483) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace
2022-11-29T11:01:56.335Z [DEBUG] provider.terraform-provider-databricks_v1.6.5: 400 Bad Request {
2022-11-29T11:01:56.335Z [WARN]  provider.terraform-provider-databricks_v1.6.5: /api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/4223940169875351:400 - Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/4223940169875351) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace: timestamp=2022-11-29T11:01:56.335Z
2022-11-29T11:01:56.335Z [WARN]  provider.terraform-provider-databricks_v1.6.5: /api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/4223940169875351:400 - Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/4223940169875351) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace: timestamp=2022-11-29T11:01:56.335Z
2022-11-29T11:01:56.335Z [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_severity=ERROR tf_req_id=c524a704-985e-7a1e-b0ab-ddfdf082684d tf_rpc=ApplyResourceChange @module=sdk.proto diagnostic_detail= diagnostic_summary="cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/4223940169875351) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace" tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/databricks/databricks tf_resource_type=databricks_mws_permission_assignment timestamp=2022-11-29T11:01:56.335Z
2022-11-29T11:01:56.336Z [ERROR] vertex "module.production.databricks_mws_permission_assignment.sp" error: cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/4223940169875351) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace
2022-11-29T11:01:56.420Z [DEBUG] provider.terraform-provider-databricks_v1.6.5: 400 Bad Request {
2022-11-29T11:01:56.420Z [WARN]  provider.terraform-provider-databricks_v1.6.5: /api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/38950186927645:400 - Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/38950186927645) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace: timestamp=2022-11-29T11:01:56.420Z
2022-11-29T11:01:56.420Z [WARN]  provider.terraform-provider-databricks_v1.6.5: /api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/38950186927645:400 - Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/38950186927645) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace: timestamp=2022-11-29T11:01:56.420Z
2022-11-29T11:01:56.421Z [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_detail= diagnostic_severity=ERROR diagnostic_summary="cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/38950186927645) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace" tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=11affbed-6b6a-7acb-6ee3-a56f48a3e040 tf_resource_type=databricks_mws_permission_assignment tf_rpc=ApplyResourceChange timestamp=2022-11-29T11:01:56.420Z
2022-11-29T11:01:56.422Z [ERROR] vertex "module.production.databricks_mws_permission_assignment.silver" error: cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/38950186927645) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace
2022-11-29T11:01:56.445Z [DEBUG] provider.terraform-provider-databricks_v1.6.5: 400 Bad Request {
2022-11-29T11:01:56.445Z [WARN]  provider.terraform-provider-databricks_v1.6.5: /api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/821050445512535:400 - Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/821050445512535) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace: timestamp=2022-11-29T11:01:56.445Z
2022-11-29T11:01:56.445Z [WARN]  provider.terraform-provider-databricks_v1.6.5: /api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/821050445512535:400 - Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/821050445512535) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace: timestamp=2022-11-29T11:01:56.445Z
2022-11-29T11:01:56.446Z [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto tf_rpc=ApplyResourceChange tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=c770f6c3-5ed6-bb53-b21e-e20c080e38b3 diagnostic_detail= diagnostic_severity=ERROR diagnostic_summary="cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/821050445512535) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace" tf_proto_version=5.3 tf_resource_type=databricks_mws_permission_assignment timestamp=2022-11-29T11:01:56.445Z
2022-11-29T11:01:56.449Z [ERROR] vertex "module.production.databricks_mws_permission_assignment.add_metastore_admins" error: cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/821050445512535) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace
2022-11-29T11:01:56.498Z [DEBUG] provider.terraform-provider-databricks_v1.6.5: 400 Bad Request {
2022-11-29T11:01:56.498Z [WARN]  provider.terraform-provider-databricks_v1.6.5: /api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/49272907450781:400 - Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/49272907450781) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace: timestamp=2022-11-29T11:01:56.498Z
2022-11-29T11:01:56.498Z [WARN]  provider.terraform-provider-databricks_v1.6.5: /api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/49272907450781:400 - Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/49272907450781) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace: timestamp=2022-11-29T11:01:56.498Z
2022-11-29T11:01:56.498Z [ERROR] provider.terraform-provider-databricks_v1.6.5: Response contains error diagnostic: diagnostic_summary="cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/49272907450781) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace" tf_proto_version=5.3 tf_resource_type=databricks_mws_permission_assignment diagnostic_detail= @module=sdk.proto diagnostic_severity=ERROR tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=e7b6732e-354a-96f6-a927-59ba309708a3 tf_rpc=ApplyResourceChange @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 timestamp=2022-11-29T11:01:56.498Z
2022-11-29T11:01:56.499Z [ERROR] vertex "module.production.databricks_mws_permission_assignment.gold" error: cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/49272907450781) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace

Error: cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/821050445512535) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace
  with module.production.databricks_mws_permission_assignment.add_metastore_admins,
  on modules/aws_workspace/db_workspaces.tf line 17, in resource "databricks_mws_permission_assignment" "add_metastore_admins":
  17: resource "databricks_mws_permission_assignment" "add_metastore_admins" {
Error: cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/1085788622452483) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace
  with module.production.databricks_mws_permission_assignment.bronze,
  on modules/aws_workspace/db_workspaces.tf line 29, in resource "databricks_mws_permission_assignment" "bronze":
  29: resource "databricks_mws_permission_assignment" "bronze" {
Error: cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/38950186927645) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace
  with module.production.databricks_mws_permission_assignment.silver,
  on modules/aws_workspace/db_workspaces.tf line 40, in resource "databricks_mws_permission_assignment" "silver":
  40: resource "databricks_mws_permission_assignment" "silver" {
Error: cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/49272907450781) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace
  with module.production.databricks_mws_permission_assignment.gold,
  on modules/aws_workspace/db_workspaces.tf line 51, in resource "databricks_mws_permission_assignment" "gold":
  51: resource "databricks_mws_permission_assignment" "gold" {
Error: cannot create mws permission assignment: Databricks API (/api/2.0/preview/accounts/ACCOUNT_ID/workspaces/8305471331990263/permissionassignments/principals/4223940169875351) requires you to set `host` property (or DATABRICKS_HOST env variable) to result of `databricks_mws_workspaces.this.workspace_url`. This error may happen if you're using provider in both normal and multiworkspace mode. Please refactor your code into different modules. Runnable example that we use for integration testing can be found in this repository at https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/aws-workspace
  with module.production.databricks_mws_permission_assignment.sp,
  on modules/aws_workspace/db_workspaces.tf line 61, in resource "databricks_mws_permission_assignment" "sp":
  61: resource "databricks_mws_permission_assignment" "sp" {
path=.terraform/providers/registry.terraform.io/databricks/databricks/1.6.5/linux_amd64/terraform-provider-databricks_v1.6.5 pid=18243

Important Factoids

No, but happy to chat/talk through any questions are aren't captured here.

nkvuong commented 1 year ago

This is similar to #1793, where the correct error message is not surfaced up

KingMichaelPark commented 1 year ago

The suggested error message in #1793 doesn't seem to apply though as the groups don't already exist in the workspace? I did see that ticket so I am not sure if that means you have a suggestion @nkvuong ?

nkvuong commented 1 year ago

the similarity with the other issue is that the error from the API response is suppressed by the provider and a generic error message about account API is provided instead

The actual error responded from the API will be in the debug log, but further up

If you manually build the provider from master, and re-run this, it should surface the correct error

KingMichaelPark commented 1 year ago

Ah I see, thank you, I will give that a go, I will leave the ticket open though because it still may be valid

KingMichaelPark commented 1 year ago

Interesting! Using the most recent commit provider has given me a different error I will have to dig into. Thank you @nkvuong

╷
│ Error: cannot create mws permission assignment: Permission assignment APIs are not available for this workspace.
│
│   with module.production.databricks_mws_permission_assignment.add_metastore_admins,
│   on modules/aws_workspace/db_workspaces.tf line 17, in resource "databricks_mws_permission_assignment" "add_metastore_admins":
│   17: resource "databricks_mws_permission_assignment" "add_metastore_admins" {
│
╵

KingMichaelPark commented 1 year ago

The answer seems to be that there are a few steps required to add users to a workspace. You can't add them to a workspace if the workspace itself does not have a unity catalog metastore. The metastore must be created within a workspace.

So one must create a workspace in one module using the mws provider
The databricks.workspace provider must be used to create the metastore
The metastore aspect of the workspace should be in a separate module to other workspace related resources like clusters and jobs because each metastore has to be specific to one Aws region. You can't have more than one metastore in a single region.
Once created, the metastore id and group attachments can be done in a third/fourth module which is specific to individual workspace resources

databricks / terraform-provider-databricks