search

databricks-industry-solutions / security-analysis-tool

Security Analysis Tool (SAT) analyzes customer's Databricks account and workspace security configurations and provides recommendations that help them follow Databrick's security best practices. When a customer runs SAT, it will compare their workspace configurations against a set of security best practices and delivers a report.
Other
86 stars 39 forks source link

Support creating tables in Unity Catalog #39

Closed zcking closed 4 months ago

zcking commented 1 year ago

I'd like to add another variable to the Terraform that allows users to have the tables created in a Unity Catalog catalog. This can be done with minimal change by simply adding the following Spark option to the new_cluster blocks in Terraform:

spark_conf = {
  "spark.databricks.sql.initial.catalog.name" = var.catalog_name
}

Then creating the new Terraform variable:

variable "catalog_name" {
  type        = string
  default     = "hive_metastore"
  description = "Default catalog to define tables into. Change this if using Unity Catalog"
}

I haven't found as simple of a way to update the dashboard yet and am open to suggestions.

At least this is how I was able to do it locally; I'd be happy to open a pull request if accepted.

arunpamulapati commented 1 year ago

Thank you. We will review and update very soon.

raghuvalusa2108 commented 4 months ago

Any update on the above? All our databricks workspaces are enabled with UC, and leveraging the code as in the current state is creating the security-analysis schema and tables under hive_metastore; Don't see an option of creating the tables under UC catalog

arunpamulapati commented 4 months ago

Yes, you just need to update the /notebooks/Utils/initialize file as noted in the Step 8 in this doc: https://github.com/databricks-industry-solutions/security-analysis-tool/blob/main/docs/setup.md

"analysis_schema_name":"security_analysis", <- database for SAT, use "catalog.database" for Unity Catalog as a store for SAT

by default SAT uses Hive metastore to store SAT analysis as tables. You can use the Unity catalog instead of using Hive metastore. Make sure the clusters have the necessary permissions when using the Unity Catalog. The CREATE SCHEMA and USE CATALOG are required so that SAT can create the Schema in the catalog.

We are releasing a much easier way to set SAT in the next couple of weeks where this edit will not be necessary as the setup process will prompt to take this as an input.