azurerm_app_configuration replica block needs to include private ip info when private endpoint associated

sdecker commented 3 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Description

Scenario: provision an azurerm_app_configuration including 2 replicas with a azurerm_private_endpoint without private DNS zone integration (We cannot use private DNS zones because of a double hub and spoke design) Private Link and DNS integration in hub and spoke network architectures

This scenario requires manual maintenance/updates of the Private Link DNS record set in every region as there is currently no automated lifecycle management for these."). On initial creation it is possible to create the azurerm_private_dns_a_record resources for each replicate private IP by iterating the custom_dns_configs of the private endpoint.

Now add a 3rd replica to the app configuration. The only change Terraform detects is to azurerm_app_configuration resource because the Private Endpoint is updated behind the scenes. The replica attribute of azurerm_app_configuration only contains ID and enpoint (string public URL). So it is not possible to add the required 3rd private DNS record because custom_dns_configs attribute of azurerm_private_endpoint has not updated.

Make zero changes but TF a second time. The azurerm_private_endpoint resources now detects the new IP and new private DNS record is created.

So the azurerm_app_configuration resources needs to return the private IPs for the replicas.

New or Affected Resource(s)/Data Source(s)

azurerm_app_configuration

Potential Terraform Configuration

replica attribute returned when there are private endpoints
replica {
  endpoint         = "https://cfg-example-replica1.azconfig.io"
  id               = "/subscriptions/xxxxxxxx/...."
  location         = "westus"
  name             = "replica1"
  private_endpoints = [
    {
        name       = "cfg-example.pe"
        private_ip = "10.72.110.28"
    }, 
    {
        name       = "cfg-example.secondary.pe"
        private_ip = "10.76.110.28"
    }
  ]
}

replica attribute returned when there are no private endpoints
replica {
  endpoint         = "https://cfg-example-replica1.azconfig.io"
  id               = "/subscriptions/xxxxxxxx/...."
  location         = "westus"
  name             = "replica1"
  private_endpoints = []
}

References

No response

teowa commented 3 months ago

Hi @sdecker , do you mean that we should export private_endpoints for the replica if the parent App Configuration has private endpoint enabled? I have checked the Azure REST API, seems we cannot retrieve the private_ip from App Configuration or Replica API. Could you share how we can get the private ip connected to the replica?

Thanks.

sdecker commented 3 months ago

Hi @teowa, unless there is another way to get Terraform to know the PE resources are changing and not require 2 deploys every time a replica is added or removed. It's a tricky issue given the behind-the-scenes design of the replicas and how PEs are updated with them. I see there is a Private EndPoint Connections under the App Configuration API that gives the ID of the endpoint associated.

sdecker commented 1 month ago

Another thought is a new resource similar to https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/cdn_frontdoor_custom_domain_association is needed for this use case? That resource forces Terraform to validate changes between two resources routes and custom domains. Perhaps something similar here for replicas and PEs?

hashicorp / terraform-provider-azurerm