Support for Azure Databricks Workspace Enhanced Security & Compliance

[zogamorph]

Is there an existing issue for this?

• [X] I have searched the existing issues

Community Note

• Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Description

The azurerm_databricks_workspace to support the enhanced security compliance options for azure. The follow options are

• Enable compliance security profile
• Apply compliance standards
• Enable enhanced security monitoring
• Enable automatic cluster update

New or Affected Resource(s)/Data Source(s)

azurerm_databricks_workspace

Potential Terraform Configuration

resource "azurerm_databricks_workspace" "myDataBricksWorkspave" {
    name                                                = "dbw-myworkspave-dev"
    resource_group_name                                 = "rg-dbw-dev"
    location                                            = "uksouth"
    sku                                                 = "premium"

    custom_parameters {
      no_public_ip                                         = true
      private_subnet_name                                  = "sub-dbw-private-dev"
      public_subnet_name                                   = "sub-dbw-public-dev"
      virtual_network_id                                   = "/resourceGroups/rg-network-dev/providers/Microsoft.Network/virtualNetworks/vnet-myvnet-DEV"
    }

   enhanced_security_compliance {
          automatic_cluster_Update = true          
          enhancedSecurityMonitoring = true          
          compliance_securit_profile  {
              complianceStandards = ["HIPAA", "PCI_DSS"]
          }
  }

References

This show how to apply the changes via the ARM template: https://learn.microsoft.com/en-us/azure/databricks/security/privacy/enhanced-security-compliance#--use-an-arm-template

[snacks-lord]

This would be extremely helpful. Currently the only option in Azure is to go into the portal to enable these (after the workspace is created).

[ssouthcity]

Once enabled in the GUI the Azure API expects these options to be sent with every subsequent update request, which breaks the functionality of our CI/CD pipelines. I have began work on this issue in my own fork of the repo :)