Error `ManagedEnvironmentInvalidNetworkConfiguration` when creating internal Azure Container App Environment

Please provide us with the following information:

This issue is a: (mark with an x)

[x] bug report -> please search issues before submitting
[ ] documentation issue or request
[ ] regression (a behavior that used to work and stopped in a new release)

Issue description

I get the following error message from Terraform:

│ Error: creating Managed Environment (Subscription: "xxx-xxx-xxx-xxx-xxx"
│ Resource Group Name: "timur-aca-test-rg"
│ Managed Environment Name: "timur-aca-test-aca-failing"): performing CreateOrUpdate: unexpected status 400 (400 Bad Request) with error: ManagedEnvironmentInvalidNetworkConfiguration: The environment network configuration is invalid: The subnet and its addressPrefix could not be found.
│ 
│   with azurerm_container_app_environment.failing_aca,
│   on main.tf line 87, in resource "azurerm_container_app_environment" "failing_aca":
│   87: resource "azurerm_container_app_environment" "failing_aca" {
│ 
│ creating Managed Environment (Subscription: "xxx-xxx-xxx-xxx-xxx"
│ Resource Group Name: "timur-aca-test-rg"
│ Managed Environment Name: "timur-aca-test-aca-failing"): performing CreateOrUpdate: unexpected status 400 (400 Bad Request) with error: ManagedEnvironmentInvalidNetworkConfiguration: The environment network configuration is invalid:
│ The subnet and its addressPrefix could not be found.

Steps to reproduce

run the following terraform code:

## Variables
variable "project" {
  description = "Identifies project(s) that the resource supports"
  type        = string
  default     = "aca-vnet-integration-test"
}

variable "azure_region" {
  description = "Azure region"
  type        = string
  default     = "australiaeast"
}

variable "tenant_id" {
  description = "Azure tenant ID"
  type        = string
  default     = "xxx-xxx-xxx-xxx-xxx"
}

## Providers
provider "azurerm" {
  features {}
  tenant_id       = var.tenant_id
  subscription_id = "xxx-xxx-xxx-xxx-xxx"
}

## Versions

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.0"
    }
  }
}

## RG
resource "azurerm_resource_group" "rg" {
  name     = "${var.project}-rg"
  location = var.azure_region
}

# VNET & two subnets, both delegated to Microsoft.App/environments
module "avm-res-network-virtualnetwork" {
  source = "Azure/avm-res-network-virtualnetwork/azurerm"

  address_space      = ["10.11.0.0/16"]
  location            = azurerm_resource_group.rg.location
  name                = "${var.project}-vnet"
  resource_group_name = azurerm_resource_group.rg.name
  enable_telemetry = false
  subnets = {
    "failing_subnet" = {
      name             = "avm-${var.project}-aca-subnet"
      address_prefixes = ["10.11.0.0/24"]
      default_outbound_access_enabled = true
      delegation = [{
        name = "fail_aca_delegation"
        service_delegation = {
          name    = "Microsoft.App/environments"
          actions = ["Microsoft.Network/virtualNetworks/subnets/join/action"]
        }
      }]
    }
  }
}

resource "azurerm_subnet" "working_subnet" {
  name                 = "azurerm-${var.project}-aca-subnet"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = module.avm-res-network-virtualnetwork.name
  address_prefixes     = ["10.11.1.0/24"]

  delegation {
    name = "working_aca_delegation"
    service_delegation {
      name    = "Microsoft.App/environments"
      actions = [ "Microsoft.Network/virtualNetworks/subnets/join/action" ]
    }
  }
}

## Azure Container Apps. One integrated into each subnet from above
resource "azurerm_container_app_environment" "failing_aca" {
  name                       = "${var.project}-aca-failing"
  location                   = azurerm_resource_group.rg.location
  resource_group_name        = azurerm_resource_group.rg.name

  infrastructure_resource_group_name = "${var.project}-aca-managed-failing-rg"
  infrastructure_subnet_id           = module.avm-res-network-virtualnetwork.subnets["failing_subnet"].resource_id
  internal_load_balancer_enabled = true

  workload_profile {
    name = "Consumption"
    workload_profile_type  = "Consumption"
    maximum_count = 3
    minimum_count = 0
  }
  zone_redundancy_enabled = true
}

resource "azurerm_container_app_environment" "working_aca" {
  name                       = "${var.project}-aca-working"
  location                   = azurerm_resource_group.rg.location
  resource_group_name        = azurerm_resource_group.rg.name

  infrastructure_resource_group_name = "${var.project}-aca-managed-working-rg"
  infrastructure_subnet_id           = azurerm_subnet.working_subnet.id
  internal_load_balancer_enabled = true

  workload_profile {
    name = "Consumption"
    workload_profile_type  = "Consumption"
    maximum_count = 3
    minimum_count = 0
  }
  zone_redundancy_enabled = true
}

Expected behavior both environments are created successfully

Actual behavior one environment gets provisioned while the other one fails with the ManagedEnvironmentInvalidNetworkConfiguration error

Screenshots
upon inspecting the two provisioned subnets with az network vnet subnet show, I see a few differences but one particularly caught my eye:

Additional context

@simonjj we use Azure/avm-res-network-virtualnetwork as part of LZ vending code (that is mostly based on Azure/terraform-azurerm-lz-vending). I can confirm we were able to successfully stand up AKS service integrated into subnet using the following code:

module "avm-res-network-virtualnetwork" {
  source = "Azure/avm-res-network-virtualnetwork/azurerm"

  address_space      = ["10.51.16.0/20"]
  location            = azurerm_resource_group.rg.location
  name                = module.naming.virtual_network.name
  resource_group_name = azurerm_resource_group.rg.name
  subnets = {
    "aks_default_node_pool_subnet" = {
      name             = "${var.project}-aks-subnet"
      address_prefixes = ["10.51.16.0/23"]
      private_link_service_network_policies_enabled = false
    }
  }
}

resource "azurerm_kubernetes_cluster" "example" {
  name                = module.naming.kubernetes_cluster.name_unique
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  // abbreviated for readability

  default_node_pool {
    name       = "default"
    // abbreviated for readability
    vnet_subnet_id = module.avm-res-network-virtualnetwork.subnets["aks_default_node_pool_subnet"].resource.id
  }  
  // abbreviated for readability
}

that said, I have just got response on the other issue I raised with AVM module repo and it seems they have enabled a workaround by exposing addressPrefix property: https://github.com/Azure/terraform-azurerm-avm-res-network-virtualnetwork/issues/101. This will work for us, but I think the root cause must have to do with the difference between how AKS and ACA integrate into subnets

microsoft / azure-container-apps