Container App with Prtivate Link in FD

[vc2023]

Describe the bug I'm trying to create a container app environment and also a private link in order to be able then to add a container with Ingress traffic limited to VNet to front door, so that container app is publicly accessible over front door. But I have issues with linking the load balancer of the container app environment with private link. Here is how I'm trying to do it in bicep files: container-app-environment.bicep

param location string = resourceGroup().location
param tags object
param infrastructureSubnetId string
param name string
param logAnalyticsWorkspaceName  string

resource logAnalytics 'Microsoft.OperationalInsights/workspaces@2020-10-01' existing = {
  name: logAnalyticsWorkspaceName
}

resource managedEnvironment 'Microsoft.App/managedEnvironments@2022-11-01-preview' = {
  name: name
  location: location
  tags: tags
  //kind: 'string'
  properties: {
    appLogsConfiguration: {
      destination: 'log-analytics'
      logAnalyticsConfiguration: {
        customerId: logAnalytics.properties.customerId
        sharedKey: logAnalytics.listKeys().primarySharedKey
      }
    }
    vnetConfiguration: {
      infrastructureSubnetId: infrastructureSubnetId
      internal: true
    }
  }
}

output managedEnvironmentId string = managedEnvironment.id
output staticIp string = managedEnvironment.properties.staticIp
output name string = managedEnvironment.name
output infrastructureResourceGroup string = managedEnvironment.properties.infrastructureResourceGroup

load-balancer.bicep

param infrastructureResourceGroup string
resource ilb 'Microsoft.Network/loadBalancers@2023-09-01' existing = {
  name:'kubernetes-internal'
  scope:resourceGroup(infrastructureResourceGroup)
}

output loadBalancerFipId string = ilb.properties.frontendIPConfigurations[0].id

private-link.bicep

param location string
param privatelinkServiceName string
param tags object
param infrastructureSubnetId string
param loadBalancerFipId string

resource privatelinkService 'Microsoft.Network/privateLinkServices@2021-05-01' = {
  name: privatelinkServiceName
  location: location
  tags: tags
  properties: {
    enableProxyProtocol: false
    loadBalancerFrontendIpConfigurations: [
      {
        id: loadBalancerFipId
      }
    ]
    ipConfigurations: [
      {
        name: 'snet-provider-default-1'
        properties: {
          privateIPAllocationMethod: 'Dynamic'
          privateIPAddressVersion: 'IPv4'
          subnet: {
            id: infrastructureSubnetId
          }
          primary: false
        }
      }
    ]
  }
}

main.bicep

module containerAppEnvironment './services/container-app-environment.bicep' = {
  name: 'container-app-environment-${envName}-internal'
  params: {
      location:location
      tags:commonTags
      name:containerAppEnvironmentNameInternal
      infrastructureSubnetId:virtualNetwork.outputs.infrastructureSubnetId
      logAnalyticsWorkspaceName:logAnalyticsWorkspace.outputs.name
  }
}

module lbi 'services/load-balancer.bicep' = {
  name:'lbi--${envName}'
  params:{
    infrastructureResourceGroup:containerAppEnvironment.outputs.infrastructureResourceGroup
  }
}

module privateLink './services/private-link.bicep' = {
  name: 'private-link-${envName}'
  params: {
      location:location
      tags:commonTags
      privatelinkServiceName:privateLinkName
      infrastructureSubnetId:virtualNetwork.outputs.infrastructureSubnetId
      loadBalancerFipId:lbi.outputs.loadBalancerFipId
  }
}

But get the error at deployment:

{"code":"DeploymentOutputEvaluationFailed","target":"/subscriptions/9e3423d5-9fb5-4d91-a647-edf09b1bfc82/resourceGroups/rg-res-dev/providers/Microsoft.Resources/deployments/container-app-environment-dev","message":"Unable to evaluate template outputs: 'infrastructureResourceGroup'. Please see error details and deployment operations. Please see https://aka.ms/arm-common-errors for usage details.","details":[{"code":"DeploymentOutputEvaluationFailed","target":"infrastructureResourceGroup","message":"The template output 'infrastructureResourceGroup' is not valid: The provided value for the template output 'infrastructureResourceGroup' is not valid. Expected a value of type 'String, Uri', but received a value of type 'Null'. Please see https://aka.ms/arm-create-parameter-file for usage details.."}]}

if I use the name of auto generated resource group instead, the deployment of resources works

module lbi 'services/load-balancer.bicep' = {
  name:'lbi--${envName}'
  params:{
    infrastructureResourceGroup:'MC_blackdesert-d2426b80-rg_blackdesert-d2426b80_northeurope'
  }
}

of course after commenting output infrastructureResourceGroup string = managedEnvironment.properties.infrastructureResourceGroup in container-app-environment.bicep How else to environment's load balancer's resource group automatically? I mean managedEnvironment.properties.infrastructureResourceGroup Thank you.

[ahelland]

You're using an older API version for your Container Environment. I know infra-rg is a property that has changed from the beginning, but I don't remember the details.

You will also probably want to use workload profile for better network support.

MS has a landing zone accelerator for a use case similar to yours so you can take a look at that if you haven't already: https://github.com/Azure/aca-landing-zone-accelerator/tree/main/scenarios/aca-internal

[vc2023]

Hi @ahelland Have changed the API version to the latest suggested, but still get the same error

param location string = resourceGroup().location
param tags object
param infrastructureSubnetId string
param name string
param logAnalyticsWorkspaceName  string

resource logAnalytics 'Microsoft.OperationalInsights/workspaces@2023-09-01' existing = {
  name: logAnalyticsWorkspaceName
}

resource managedEnvironment 'Microsoft.App/managedEnvironments@2024-03-01' = {
  name: name
  location: location
  tags: tags
  //kind: 'string'
  properties: {
    appLogsConfiguration: {
      destination: 'log-analytics'
      logAnalyticsConfiguration: {
        customerId: logAnalytics.properties.customerId
        sharedKey: logAnalytics.listKeys().primarySharedKey
      }
    }
    vnetConfiguration: {
      //dockerBridgeCidr: 'string'
      infrastructureSubnetId: infrastructureSubnetId
      internal: true
      // platformReservedCidr: 'string'
      // platformReservedDnsIP: 'string'
    }
    // workloadProfiles: [
    //   {
    //     maximumCount: int
    //     minimumCount: int
    //     name: 'string'
    //     workloadProfileType: 'string'
    //   }
    // ]
    // zoneRedundant: bool
  }
}

output managedEnvironmentId string = managedEnvironment.id
output staticIp string = managedEnvironment.properties.staticIp
output name string = managedEnvironment.name
output infrastructureResourceGroup string = managedEnvironment.properties.infrastructureResourceGroup

do I need to use workload profile? I guess not, to fix this problem.

[stephaniezyen]

Unfortunately this is an RP issue, I would open a support ticket with the Microsoft.App RP team to get this fixed - for now we are moving this issue to the repo tracking known RP issues

[ahelland]

You don't need the workload profile for this. But since you want to control ingress I thought you mind be interested in controling egress as well and pipe through UDRs or a NAT Gateway and that requires workload profiles (which can be a consumption profile in turn) :)

Btw: you get a load balancer by default with the environment. And with Application Gateway I have no problem using that. But I haven't tested Front Door with ACA. (Which is why I included the link to the landing zone accelerator.)