hashicorp / terraform-provider-azurerm

Terraform provider for Azure Resource Manager
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs
Mozilla Public License 2.0
4.51k stars 4.6k forks source link

Issues with making changes to output with Keyvault and Private Link #22792

Open ChrisCalzaretta opened 1 year ago

ChrisCalzaretta commented 1 year ago

Is there an existing issue for this?

Community Note

We are trying to change anything with private link resource on the keyvault network.

Terraform Version

1.5

AzureRM Provider Version

latest

Affected Resource(s)/Data Source(s)

azurerm, azuread

Terraform Configuration Files

private endpoint

# Create a Private Endpoint
resource "azurerm_private_endpoint" "endpoint" {
  name                = var.endpoint_name
  location            = var.resource_group_location
  resource_group_name = var.resource_group_name
  subnet_id           = var.subnet_id
  private_service_connection {
    name                           = "psc-${var.endpoint_name}"
    is_manual_connection           = "false"
    private_connection_resource_id = var.resource_id
    subresource_names              = var.subresource_type
  }
}
# Private Endpoint Connecton
data "azurerm_private_endpoint_connection" "endpoint-connection" {
  depends_on          = [azurerm_private_endpoint.endpoint]
  name                = azurerm_private_endpoint.endpoint.name
  resource_group_name = var.resource_group_name
}

# Create a Private DNS Zone - moved to shared services main
# resource "azurerm_private_dns_zone" "endpoint-dns-private-zone" {
#   name                = var.dns_zone
#   resource_group_name = var.resource_group_name
# }

# Create a Private DNS A Record
resource "azurerm_private_dns_a_record" "endpoint-dns-a-record" {
  name                = lower(var.resource_name)
  zone_name           = var.dns_zone
  resource_group_name = var.dns_zone_rg_name #resource group where dns zone exists
  ttl                 = 300
  records             = [data.azurerm_private_endpoint_connection.endpoint-connection.private_service_connection.0.private_ip_address]
}

# Create a Private DNS A Record .scm
resource "azurerm_private_dns_a_record" "endpoint-dns-a-record-scm" {
  count               = (var.isAppService==0?0:1)
  name                = "${lower(var.resource_name)}.scm"
  zone_name           = var.dns_zone
  resource_group_name = var.dns_zone_rg_name #resource group where dns zone exists
  ttl                 = 300
  records             = [data.azurerm_private_endpoint_connection.endpoint-connection.private_service_connection.0.private_ip_address]
}

# Create a Private DNS to VNET link
resource "azurerm_private_dns_zone_virtual_network_link" "dns-zone-to-vnet-link" {
  count                 = (var.dnsLink==1?1:0)
  name                  = "${var.endpoint_name}-vnet-link"
  resource_group_name   = var.dns_zone_rg_name #resource group where dns zone exists
  private_dns_zone_name = var.dns_zone
  virtual_network_id    = var.vnet_id #vnet ID of the new stack
}

Outputs
# output "Resource_Group_ID" {
#   value = azurerm_resource_group.rg.id
# }
# output "Resource_Group_Name" {
#   value = var.resource_group_name
# }

output "private_ip_address" {
  value = azurerm_private_endpoint.endpoint.private_service_connection[0].private_ip_address
}

output "fqdn" {
  value = azurerm_private_dns_a_record.endpoint-dns-a-record.fqdn
}

Debug Output/Panic Output

2023-08-02T15:22:19.8480516Z Error: retrieving `contact` for KeyVault: keyvault.BaseClient#GetCertificateContacts: Failure sending request: StatusCode=0 -- Original Error: context deadline exceeded
2023-08-02T15:22:19.8480862Z
2023-08-02T15:22:19.8481275Z   with module.Key_Vault.azurerm_key_vault.key_vault,
2023-08-02T15:22:19.8481666Z   on ../../modules/KeyVault/main.tf line 2, in resource "azurerm_key_vault" "key_vault":
2023-08-02T15:22:19.8482024Z    2: resource "azurerm_key_vault" "key_vault" {
2023-08-02T15:22:19.8482256Z
2023-08-02T15:22:19.8999761Z [32;1mPath[0m
2023-08-02T15:22:19.9000673Z [32;1m----[0m

Expected Behaviour

that it would update the output

Actual Behaviour

while reading the state the error happens.. we dont get to the plan

Steps to Reproduce

terraform init works terraform plan fails

Important Factoids

No response

References

No response

magodo commented 1 year ago

This duplicates to https://github.com/hashicorp/terraform-provider-azurerm/issues/17863. The error presumably indicates a DNS resolve error, and please ensure your environment (running terraform) can access the subnet specified in the PE, and no NSR (or the alikes) to block the access.

w4rgrum commented 10 months ago

As said in this comment, I think the azurerm provider should avoid touching the dataplane if not required (i.e. if your are not manipulating actual data). The reason behind this is that your deployment infra should not have permissions to reach the dataplane unless necessary. Also any misconfiguration will lead to a dead state as your KV resource cannot be refreshed anymore.

Additionally, this certificates fetching seems to have been designed as a best-effort as if 403/404 are received it is ignored, thus seems legit to have it ignored in case of resolution failure, meaning the access to the KV is also denied to the user/SP in some way.

exe-r commented 4 months ago

Still relevant today, worst part as a comment above is that your state file will always fail as it can't reach this data place. Manual steps are then needed to resolve, becomes very confusing with pipeline runs failing.