Open igor-marinho opened 1 year ago
Thanks for raising this issue. Seems I cannot repro this issue with below tf config that is similar with yours. Could you try below tf config to see if the issue still exists? If no, could you compare below tf config with yours to figure out the difference? Thanks.
provider "azurerm" {
features {
key_vault {
purge_soft_delete_on_destroy = false
purge_soft_deleted_keys_on_destroy = false
}
}
}
data "azurerm_client_config" "current" {}
resource "azurerm_resource_group" "test" {
name = "acctestRG-mysql-test0136"
location = "west europe"
}
resource "azurerm_virtual_network" "test" {
name = "acctest-vn-test0136"
location = azurerm_resource_group.test.location
resource_group_name = azurerm_resource_group.test.name
address_space = ["10.0.0.0/16"]
}
resource "azurerm_subnet" "test" {
name = "acctest-sn-test0136"
resource_group_name = azurerm_resource_group.test.name
virtual_network_name = azurerm_virtual_network.test.name
address_prefixes = ["10.0.2.0/24"]
service_endpoints = ["Microsoft.Storage"]
delegation {
name = "fs"
service_delegation {
name = "Microsoft.DBforMySQL/flexibleServers"
actions = [
"Microsoft.Network/virtualNetworks/subnets/join/action",
]
}
}
}
resource "azurerm_private_dns_zone" "test" {
name = "acctest0136.mysql.database.azure.com"
resource_group_name = azurerm_resource_group.test.name
}
resource "azurerm_private_dns_zone_virtual_network_link" "test" {
name = "acctestVnetZonetest0136.com"
private_dns_zone_name = azurerm_private_dns_zone.test.name
virtual_network_id = azurerm_virtual_network.test.id
resource_group_name = azurerm_resource_group.test.name
}
resource "azurerm_user_assigned_identity" "test" {
name = "acctestmitest0136"
location = azurerm_resource_group.test.location
resource_group_name = azurerm_resource_group.test.name
}
resource "azurerm_key_vault" "test" {
name = "acctestkvtest0136"
location = azurerm_resource_group.test.location
resource_group_name = azurerm_resource_group.test.name
tenant_id = data.azurerm_client_config.current.tenant_id
sku_name = "standard"
purge_protection_enabled = true
}
resource "azurerm_key_vault_access_policy" "server" {
key_vault_id = azurerm_key_vault.test.id
tenant_id = data.azurerm_client_config.current.tenant_id
object_id = azurerm_user_assigned_identity.test.principal_id
key_permissions = ["Get", "List", "WrapKey", "UnwrapKey", "GetRotationPolicy", "SetRotationPolicy"]
}
resource "azurerm_key_vault_access_policy" "client" {
key_vault_id = azurerm_key_vault.test.id
tenant_id = data.azurerm_client_config.current.tenant_id
object_id = data.azurerm_client_config.current.object_id
key_permissions = ["Get", "Create", "Delete", "List", "Restore", "Recover", "UnwrapKey", "WrapKey", "Purge", "Encrypt", "Decrypt", "Sign", "Verify", "GetRotationPolicy", "SetRotationPolicy"]
}
resource "azurerm_key_vault_key" "test" {
name = "test"
key_vault_id = azurerm_key_vault.test.id
key_type = "RSA"
key_size = 2048
key_opts = ["decrypt", "encrypt", "sign", "unwrapKey", "verify", "wrapKey"]
depends_on = [
azurerm_key_vault_access_policy.client,
azurerm_key_vault_access_policy.server,
]
}
resource "azurerm_mysql_flexible_server" "test" {
name = "acctest-fs-test0136"
resource_group_name = azurerm_resource_group.test.name
location = azurerm_resource_group.test.location
version = "5.7"
delegated_subnet_id = azurerm_subnet.test.id
private_dns_zone_id = azurerm_private_dns_zone.test.id
geo_redundant_backup_enabled = false
backup_retention_days = 30
administrator_login = "dbaadmin"
administrator_password = "Test##00!"
sku_name = "GP_Standard_D2ds_v4"
identity {
type = "UserAssigned"
identity_ids = [azurerm_user_assigned_identity.test.id]
}
customer_managed_key {
key_vault_key_id = azurerm_key_vault_key.test.id
primary_user_assigned_identity_id = azurerm_user_assigned_identity.test.id
}
storage {
auto_grow_enabled = false
size_gb = 20
}
lifecycle {
ignore_changes = [zone]
}
depends_on = [azurerm_private_dns_zone_virtual_network_link.test]
}
@neil-yechenwei Currently facing the same issue with a similar setup you brought up:
resource "azurerm_resource_group" "test" {
name = "rg-app-dev-euwest"
location = "west europe"
tags = merge(var.tags, { "app" : "test" })
}
resource "azurerm_mysql_flexible_server" "test" {
name = lower("mysql-app-dev-euwest") # require
location = "west europe"
resource_group_name = azurerm_resource_group.test.name
administrator_login = "app_admin"
administrator_password = random_password.test-password.result
sku_name = "GP_Standard_D2ds_v4"
version = 5.7
storage {
auto_grow_enabled = false
size_gb = 20
}
identity {
type = "UserAssigned"
identity_ids = [azurerm_user_assigned_identity.mysql.id]
}
customer_managed_key {
key_vault_key_id = data.azurerm_key_vault_key.keyvault-key.id
primary_user_assigned_identity_id = azurerm_user_assigned_identity.mysql.id
}
}
resource "azurerm_key_vault_access_policy" "test" {
key_vault_id = data.azurerm_key_vault.keyvault.id
tenant_id = data.azurerm_client_config.current.tenant_id
object_id = azurerm_user_assigned_identity.mysql.principal_id
key_permissions = ["Get", "UnwrapKey", "WrapKey"]
}
resource "azurerm_user_assigned_identity" "test" {
location = "west europe"
name = "id-app-dev-euwest"
resource_group_name = azurerm_resource_group.test.name
}
The KeyVault in that case is set in another resource Group. Maybe with that you are able to reproduce it.
Edit:
Tried it with separate KeyVault in the same resource group and worked just fine. Setup with KeyVault in different Resource Group works for flexible PostgreSQL, so it probably should for MySQL as well?
@neil-yechenwei Currently facing the same issue with a similar setup you brought up:
resource "azurerm_resource_group" "test" { name = "rg-app-dev-euwest" location = "west europe" tags = merge(var.tags, { "app" : "test" }) } resource "azurerm_mysql_flexible_server" "test" { name = lower("mysql-app-dev-euwest") # require location = "west europe" resource_group_name = azurerm_resource_group.test.name administrator_login = "app_admin" administrator_password = random_password.test-password.result sku_name = "GP_Standard_D2ds_v4" version = 5.7 storage { auto_grow_enabled = false size_gb = 20 } identity { type = "UserAssigned" identity_ids = [azurerm_user_assigned_identity.mysql.id] } customer_managed_key { key_vault_key_id = data.azurerm_key_vault_key.keyvault-key.id primary_user_assigned_identity_id = azurerm_user_assigned_identity.mysql.id } } resource "azurerm_key_vault_access_policy" "test" { key_vault_id = data.azurerm_key_vault.keyvault.id tenant_id = data.azurerm_client_config.current.tenant_id object_id = azurerm_user_assigned_identity.mysql.principal_id key_permissions = ["Get", "UnwrapKey", "WrapKey"] } resource "azurerm_user_assigned_identity" "test" { location = "west europe" name = "id-app-dev-euwest" resource_group_name = azurerm_resource_group.test.name }
The KeyVault in that case is set in another resource Group. Maybe with that you are able to reproduce it.
Edit:
Tried it with separate KeyVault in the same resource group and worked just fine. Setup with KeyVault in different Resource Group works for flexible PostgreSQL, so it probably should for MySQL as well?
The weird part for is even adding mysql-flexible in the same resource group as my keyvault it still saying it cannot find it and theyurl/ket/version are all good.
Closed it by mistake ....
I'm tried creating my own keyvault and when creating the mysql flexible and I got same result.
resource "azurerm_subnet" "main" {
provider = azurerm.current_sub
name = "mysql-flexi-subnet"
resource_group_name = data.azurerm_virtual_network.main.resource_group_name
virtual_network_name = data.azurerm_virtual_network.main.name
address_prefixes = ["${cidrsubnet(data.azurerm_virtual_network.main.address_space[0], 4, 15)}"]
service_endpoints = ["Microsoft.Storage"]
delegation {
name = "fs"
service_delegation {
name = "Microsoft.DBforMySQL/flexibleServers"
actions = [
"Microsoft.Network/virtualNetworks/subnets/join/action",
]
}
}
lifecycle {
ignore_changes = [address_prefixes]
}
}
resource "azurerm_user_assigned_identity" "main" {
name = "test-iden"
location = var.resource_location
resource_group_name = var.rg_name
}
resource "azurerm_key_vault" "main" {
name = "keyvaulttes1"
location = var.resource_location
resource_group_name = var.rg_name
tenant_id = data.azurerm_client_config.current.tenant_id
sku_name = "standard"
purge_protection_enabled = true
}
resource "azurerm_key_vault_access_policy" "server" {
key_vault_id = azurerm_key_vault.main.id
tenant_id = data.azurerm_client_config.current.tenant_id
object_id = azurerm_user_assigned_identity.main.principal_id
key_permissions = ["Get", "List", "WrapKey", "UnwrapKey", "GetRotationPolicy", "SetRotationPolicy"]
}
resource "azurerm_key_vault_access_policy" "client" {
key_vault_id = azurerm_key_vault.main.id
tenant_id = data.azurerm_client_config.current.tenant_id
object_id = data.azurerm_client_config.current.object_id
key_permissions = ["Get", "Create", "Delete", "List", "Restore", "Recover", "UnwrapKey", "WrapKey", "Purge", "Encrypt", "Decrypt", "Sign", "Verify", "GetRotationPolicy", "SetRotationPolicy"]
}
resource "azurerm_key_vault_key" "main" {
name = "main"
key_vault_id = azurerm_key_vault.main.id
key_type = "RSA"
key_size = 2048
key_opts = ["decrypt", "encrypt", "sign", "unwrapKey", "verify", "wrapKey"]
depends_on = [
azurerm_key_vault_access_policy.client,
azurerm_key_vault_access_policy.server,
]
}
resource "azurerm_mysql_flexible_server" "main" {
name = "accmain-fs-main0136"
resource_group_name = var.rg_name
location = var.resource_location
version = "5.7"
delegated_subnet_id = azurerm_subnet.main.id
private_dns_zone_id = local.private_dns_zone_id
geo_redundant_backup_enabled = false
backup_retention_days = 30
administrator_login = "dbaadmin"
administrator_password = "main##00!"
sku_name = "GP_Standard_D2ds_v4"
identity {
type = "UserAssigned"
identity_ids = [azurerm_user_assigned_identity.main.id]
}
customer_managed_key {
key_vault_key_id = azurerm_key_vault_key.main.id
primary_user_assigned_identity_id = azurerm_user_assigned_identity.main.id
}
storage {
auto_grow_enabled = false
size_gb = 20
}
lifecycle {
ignore_changes = [zone]
}
}
│ Error: creating Flexible Server (Subscription: "***" │ Resource Group Name: "na-mysql-stable-rg" │ Flexible Server Name: "accmain-fs-main0136"): polling after Create: Code="AzureKeyVaultKeyNotFound" Message="Could not find Azure Key Vault Key with key name 'https://keyvaulttes1.vault.azure.net/keys/...."
Doing some tests I've create it manually without the CMK and after I tried to enable CMK as below:
# module.root-mysql-flexi["poc"].module.mysqlflexi.azurerm_mysql_flexible_server.main will be updated in-place
~ resource "azurerm_mysql_flexible_server" "main" {
id = "/subscriptions/*************/resourceGroups/mysqlflexiac-rg/providers/Microsoft.DBforMySQL/flexibleServers/mysqlflexiac"
name = "mysqlflexiac"
# (15 unchanged attributes hidden)
+ customer_managed_key {
+ key_vault_key_id = "https://na-mysql-stable********t/keys/mysqlflexiac-cmk/********8df2b4a450a022"
+ primary_user_assigned_identity_id = "/subscriptions/************/resourceGroups/mysqlflexiac-rg/providers/Microsoft.ManagedIdentity/userAssignedIdentities/mysqlflexiac-identity-server"
}
# (2 unchanged blocks hidden)
}
Same error:
│ Error: updating Flexible Server (Subscription: "*********"
│ Resource Group Name: "mysqlflexiac-rg"
│ Flexible Server Name: "mysqlflexiac"): polling after Update: Code="AzureKeyVaultKeyNotFound" Message="Could not find Azure Key Vault Key with key name 'https://na-mysql-stable************/keys/mysqlflexiac-cmk/*********829b18df2b4a450a022'."
When I do it manually it works just fine.
So, fit seems to be a bug on the azurerm provider or in Mysql Flexible server itself.
@igor-marinho - I believe it's not a problem in a provider but rather the limitation of a CMK for MySQL as stated here: https://learn.microsoft.com/en-us/azure/mysql/flexible-server/concepts-customer-managed-key#limitations
This feature is only supported for key vaults, which allow public access from all networks.
I have the same issue trying to set encryption for MySQL in a portal. Despite setting the values I get the same error when I try to save the configuration. Can you check whether your AKV is set to "private networks access"?
@bwilczynski Yes, I use vnet integration for mysql flexible and private connection for all keyvaults. I thought it'd work asme as flexible server but is seems that it doesn't.
Support for MySQL Flexible Server CMK with VNet integration is currently being evaluated by Microsoft in Private Preview. I would advise you to wait for it to become GA.
Is there an existing issue for this?
Community Note
Terraform Version
1.3.9
AzureRM Provider Version
3.46.0
Affected Resource(s)/Data Source(s)
azurerm_mysql_flexible_server
Terraform Configuration Files
Debug Output/Panic Output
Expected Behaviour
Object cerated.
Actual Behaviour
It fails after creation because it cannot find the keyvault with the key.
Steps to Reproduce
resource "azurerm_mysql_flexible_server" "main" { ..... identity { type = "UserAssigned" identity_ids = [var.user_assigned_identity_id] }
customer_managed_key { key_vault_key_id = var.keyvault_vault_key_id primary_user_assigned_identity_id = var.user_assigned_identity_id } ..... }
Important Factoids
No response
References
No response