General Clean-Up PR

[JDBraun]

General Updates:

• Provider Upgrade: The Databricks provider has been upgraded to 1.46. No impact was seen in usage from 1.46 to the previous version. However, if you're managing an active environment, you should do your own testing in your lower environments.

• README: Moved the readme to the top level of the AWS repository for easier access, isolating the relevant Terraform code. Cleaned-up the Post Workspace Deployment to only include modules that are deployed by default. All optional modes are outlined in Optional Deployment Configuration.

• Metastore Creation:

  • Metastore ID variable has been removed. Now it is a boolean value if metastore exists or not in the region. If a metastore exists for that region, then the new workspace will be attached. This will have an impact to the very small percentage of accounts that have multiple metastores in the same region. They can directly add the metastore ID in the metastore assignment TF file. This addresses this issue #67

• Variable Updates:

  • Booleans in variables names: Any optional variable that is true or false now has the same naming convention. Like enable_read_only_external_location_boolean, enable_admin_configs_boolean, etc.
  • dbfsname variable: Removed DBFS name as variable to insert. This is now created based on the resource prefix. It is now renamed as "root_s3_bucket" throughout the codebase for additional clarity.
  • Workspace admin service principal variable: This is now includes admin in the name to represent to permission level of this created SP.
  • Compliance security profile egress ports: This has been renamed to better describe it's functionality now that a compliance security profile resource exists. This enablement only impacts the outbound ports on the workspace SG and inbound ports on the PrivateLink.
  • Error message on operation_mode: Updated the error message so it no longer says network type.

• template.tfvars.example:

  • The template file now has comments on what values to provide for each of the values for clarity.

Individual Files:

• sra.tf:

  • The main file has been cleaned up and organized based on required variables, operation mode, and the optional modules that can be enabled on a deployment.
  • Made the default amount of subnets to three instead of two to increase the high availability of the workspace, which is more applicable to production use cases.

• networking.tf:

  • VPC has been renamed from 'data plane' to 'classic compute plane' for clarity
  • Workspace security group has been renamed from 'data plane security group' to 'workspace SG' for clarity

• privatelink.tf:

  • Fixed an issue around PrivateLink endpoints not being applied to all availability zones if more than two subnets were provided in the variables.

• cmk.tf:

  • Added tags for both the workspace storage and managed key.

• credential.tf

  • Removed redundant allow block image resource with the condition key of the Databricks account ID. This is covered by the explicit denied that was added in a previous commit.

• workspace.tf

  • Added a commented out deployment_name in the workspace resource configuration. This is not a feature available to all accounts, so this can be commented out and added as a variable for applicable customers.

[JDBraun]

@metrocavich updated the dbfs_s3.tf -> root_s3_bucket.tf, I added the account ID to the bucket policy as per best practices for that, and then I updated the readme

[JDBraun]

@metrocavich - updated the sra.tf with a note about the VPC CIDR range

@ryan-gord-db - updated the variables.tf with the right operation mode, then updated the sra.tf with a note on the metastore ID

[JDBraun]

@metrocavich / @ryan-gord-db - let me know if y'all are good on this PR, or if there's anything else