search

Azure / terraform-azurerm-avm-res-storage-storageaccount

This Terraform module is designed to create Azure Storage Accounts and its related resources, including blob containers, queues, tables, and file shares. It also supports the creation of a storage account private endpoint which provides secure and direct connectivity to Azure Storage over a private network.
https://registry.terraform.io/modules/Azure/avm-res-storage-storageaccount
MIT License
19 stars 27 forks source link

Customer Managed Key is not present in Azure Portal #107

Closed theccz closed 2 months ago

theccz commented 3 months ago

Is there an existing issue for this?

Greenfield/Brownfield provisioning

greenfield

Terraform Version

1.8.4

Module Version

v0.1.2

AzureRM Provider Version

~> 3.0

Affected Resource(s)/Data Source(s)

azurerm_storage_account_customer_managed_key

Terraform Configuration Files

# Main.TF

# Manage Key Vaults based on the key_vaults map variable

module "key_vault" {
  for_each            = var.key_vaults
  source              = "github.com/Azure/terraform-azurerm-avm-res-keyvault-vault?ref=v0.5.3"
  depends_on          = [azurerm_resource_group.resource_group]
  name                = "${local.resource_full_prefix}${each.key}"
  location            = data.azurerm_resource_group.resource_group.location
  resource_group_name = data.azurerm_resource_group.resource_group.name
  tenant_id           = data.azurerm_client_config.current.tenant_id
  enable_telemetry    = false
  tags                = merge(local.common_tags, try(each.value.tags, {}))
  enabled_for_deployment          = each.value.enabled_for_deployment
  enabled_for_disk_encryption     = each.value.enabled_for_disk_encryptio
  enabled_for_template_deployment = each.value.enabled_for_template_deployment

  purge_protection_enabled        = each.value.purge_protection_enabled
  soft_delete_retention_days      = each.value.soft_delete_retention_days

  keys             = try(local.key_vaults_keys["${each.key}"], null)

  role_assignments = merge({

    for role_assignment_key, role_assignment_values in each.value.role_assignments 
    role_assignment_key => {
      role_definition_id_or_name = role_assignment_values.role_definition_id_or_name
      principal_id = (
        # Is the object an Azure principal id or object id (UUID) or a deployment resource (vm)
        can(regex("^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$", role_assignment_values.principal_id)) 
        role_assignment_values.principal_id : coalesce(

          # Is the object a deployment VM resource use the system assigned principal id

          can(regex("^vm", role_assignment_values.principal_id)) ? module.linux_vm["${role_assignment_values.principal_id}"].system_assigned_mi_principal_id : null
        )
      )

      skip_service_principal_aad_check = role_assignment_values.skip_service_principal_aad_check
    } if length(each.value.role_assignments) > 0
  }, { spn = {role_definition_id_or_name = "Key Vault Administrator", principal_id = data.azurerm_client_config.current.object_id }})

  public_network_access_enabled = false
  network_acls = {
    bypass                     = "AzureServices"
    default_action             = "Deny
    ip_rules                   = []
    virtual_network_subnet_ids = []
  }

 private_endpoints = {
    "${each.key}" = 
      name                            = "${local.resource_full_prefix_dashed}-${each.key}-pep-kvt"
      subnet_resource_id              = data.azurerm_subnet.subnet_vm.id
      private_dns_zone_resource_ids   = ["${local.private_dns_zone_subscription_id}/providers/Microsoft.Network/privateDnsZones/${local.private_link_zones["keyvault"]}"]
      private_service_connection_name = "${local.resource_full_prefix_dashed}-${each.key}-pep-kvt"
      network_interface_name          = "${local.resource_full_prefix_dashed}-${each.key}-pep-kvt-nic01"
      inherit_tags                    = false
      inherit_lock                    = false
      tags                            = local.common_tags
    }
  }
}

# Manage Storage Accounts based on storage_accounts map variable

module "storage_account" {
  for_each                      = var.storage_accounts
  depends_on                    = [azurerm_resource_group.resource_group, module.key_vault]
  source                        = "github.com/Azure/terraform-azurerm-avm-res-storage-storageaccount?ref=v0.1.2
  name                          = "${local.resource_full_prefix}${each.key}"
  resource_group_name           = data.azurerm_resource_group.resource_group.name
  tags                          = merge(local.common_tags, try(each.value.tags, {}))
  location                      = data.azurerm_resource_group.resource_group.location
  account_kind                  = each.value.account_kind
  account_replication_type      = each.value.account_replication_type
  account_tier                  = each.value.account_tier
  min_tls_version               = each.value.min_tls_version
  shared_access_key_enabled     = each.value.shared_access_key_enabled
  public_network_access_enabled = each.value.public_network_access_enabled
  infrastructure_encryption_enabled = each.value.infrastructure_encryption_enabled
  containers                    = try(local.storage_accounts_containers["${each.key}"], null)
  queues                        = try(local.storage_accounts_queues["${each.key}"], null)
  shares                        = try(local.storage_accounts_shares["${each.key}"], null)
  tables                        = try(local.storage_accounts_tables["${each.key}"], null)

  role_assignments = 
    for role_assignment_key, role_assignment_values in each.value.role_assignments :
    role_assignment_key => {
      role_definition_id_or_name = role_assignment_values.role_definition_id_or_name
      principal_id = (
        # Is the object an Azure principal id or object id (UUID) or a deployment resource (vm)
        can(regex("^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$", role_assignment_values.principal_id)) ?
        role_assignment_values.principal_id : coalesce(
          # Is the object a deployment VM resource use the system assigned principal id

          can(regex("^vm", role_assignment_values.principal_id)) ? module.linux_vm["${role_assignment_values.principal_id}"].system_assigned_mi_principal_id : null
        )
      )

      skip_service_principal_aad_check = role_assignment_values.skip_service_principal_aad_check
    } if length(each.value.role_assignments) > 0
  }

  network_rules = {
    bypass                     = ["AzureServices"]
    default_action             = "Deny"
    ip_rules                   = []
    virtual_network_subnet_ids = [
  }

  # create a private endpoint for each endpoint type

  private_endpoints = {
    for endpoint in local.storage_accounts_endpoints["${each.key}"] :
    endpoint => {
      name                            = "${local.resource_full_prefix_dashed}-${each.key}-pep-${endpoint}"
      subnet_resource_id              = data.azurerm_subnet.subnet_vm.id
      subresource_name                = endpoint
      private_dns_zone_resource_ids   = ["${local.private_dns_zone_subscription_id}/providers/Microsoft.Network/privateDnsZones/${local.private_link_zones[endpoint]}"]
      private_service_connection_name = "${local.resource_full_prefix_dashed}-${each.key}-pep-${endpoint}"
      network_interface_name          = "${local.resource_full_prefix_dashed}-${each.key}-pep-${endpoint}-nic01"
      inherit_tags                    = false
      inherit_lock                    = false
      tags                            = local.common_tags
    }
  }

  managed_identities = {
    system_assigned           = each.value.system_assigned
      }

  customer_managed_key = each.value.customer_managed_key != null ? {
    key_vault_resource_id = each.value.customer_managed_key.key_vault
    key_name              = each.value.customer_managed_key.key_name
  } : null

}

# Variables.tf

variable "key_vaults" {
  type = map(object({
    enabled_for_deployment          = optional(bool, false)
    enabled_for_disk_encryption     = optional(bool, false)
    enabled_for_template_deployment = optional(bool, false)
    keys = optional(map(object({
      key_type = string
      key_opts = optional(list(string), ["sign", "verify"])
      key_size        = optional(number, null)
      curve           = optional(string, null)
      not_before_date = optional(string, null)
      expiration_date = optional(string, null)
      tags            = optional(map(any), null)
      role_assignments = optional(map(object({
        role_definition_id_or_name             = string
        principal_id                           = string
        description                            = optional(string, null)
        skip_service_principal_aad_check       = optional(bool, false)
        condition                              = optional(string, null)
        condition_version                      = optional(string, null)
        delegated_managed_identity_resource_id = optional(string, null)
      })), {})
      rotation_policy = optional(object({
        automatic = optional(object({
          time_after_creation = optional(string, null)
          time_before_expiry  = optional(string, null)
        }), null)
        expire_after         = optional(string, null)
        notify_before_expiry = optional(string, null)
      }), null)
    })), {})
    purge_protection_enabled        = optional(bool, true)
    role_assignments                = optional(map(object({
      role_definition_id_or_name       = string
      principal_id                     = string
      skip_service_principal_aad_check = optional(bool, false)
    })), {})
    soft_delete_retention_days      = optional(number, 90)
    tags                            = optional(map(string), {})
  }))
  default = {
}

variable "storage_accounts" {
  type = map(object({
    account_kind                  = optional(string, "StorageV2")
    account_replication_type      = optional(string, "LRS")
    account_tier                  = optional(string, "Standard")
    min_tls_version               = optional(string, "TLS1_2")
    shared_access_key_enabled     = optional(bool, false
    public_network_access_enabled = optional(bool, false)
    system_assigned               = optional(bool, true)
    infrastructure_encryption_enabled = optional(bool, true
    containers = optional(map(object({
      publicAccess = optional(string, "None")
      metadata     = optional(map(string))
      role_assignments = optional(map(object({
        role_definition_id_or_name       = string
        principal_id                     = string
        skip_service_principal_aad_check = optional(bool, false)
      })), {})
    })), {})
    queues = optional(map(object({
      metadata = optional(map(string))
    })), {})
    shares = optional(map(object({
      quota            = number
      enabled_protocol = optional(string, "NFS")
      access_tier      = optional(string, "Premium")
      metadata         = optional(map(string))
    })), {})
    tables = optional(map(object({})), {})
    role_assignments = optional(map(object({
      role_definition_id_or_name       = strin
      principal_id                     = string
      skip_service_principal_aad_check = optional(bool, false)
    })), {})
    customer_managed_key = optional(object({
      key_vault             = optional(string)
      key_name              = optional(string)
      user_assigned_identity = optional(object({
        resource_id = string
      }))
    }))    
  }))

  default = {}

}

tfvars variables values

# TFVars
---
location: West Europe
environment: np1
application: test
key_vaults:
  kvt01:
    purge_protection_enabled: false
    soft_delete_retention_days: 7
    keys:
      cmk00:
        key_type: RSA
        key_size: 2048
        key_opts: [ "decrypt", "encrypt", "sign", "unwrapKey", "verify", "wrapKey"]
    role_assignments:
      test_owner
        role_definition_id_or_name: Owner
        principal_id: my_principal_id  
      test_kv_admin:
        role_definition_id_or_name: Key Vault Administrator
        principal_id: my_principal_id    
      test_kv_crypto_officer:
        role_definition_id_or_name: Key Vault Crypto Office
        principal_id: my_principal_id                    
storage_accounts:
  sts00:
    system_assigned: false
    containers:
      test1:
        metadata:
          business_data: noop
    role_assignments:
      test:
        role_definition_id_or_name: Storage Blob Data Reader
        principal_id: my_principal_id
    key_vault: kvt01
    key_name: cmk00

Debug Output/Panic Output

N/A code runs

Expected Behaviour

I should see in Azure portal customer managed key.

Actual Behaviour

However after successful TF run the storage account still has Microsoft Key. Please advise

Steps to Reproduce

No response

Important Factoids

No response

References

No response

matt-FFFFFF commented 2 months ago

Hi @theccz

We need to confirm if this is a portal bug or if the resource is actually not there.

If it's the former then we might not be able to do anything about it.

Are you able to confirm if the CMK Is actually present?

Perhaps do a az rest --method GET --uri '/subscriptions/...?api-version=aaaa-bb-cc'

chinthakaru commented 2 months ago

Hi @theccz

I was able to deploy customer_managed_key example in my tenant without any issues and i see CMK within the portal. if you are still having the issue, please can you provide full configuration files here?

microsoft-github-policy-service[bot] commented 2 months ago

[!NOTE] The "Needs: Author Feedback :ear:" label was removed and the "Needs: Attention :wave:" label was added as per ITA11.

theccz commented 2 months ago

Hello all, thanks for getting back to me. I was able to deploy CMK but only with user-managed-identites cannot do it with system-managed identites.