Cannot setup AGIC with AAD Pod Identity

[oscarlnetoo]

Hello,

I am trying to install AGIC 1.7.0 with AAD Pod Identity as ARM authentication. Unfortunately, the ingress-azure pod is crashing on startup. Could someone help me with that? I have tried everything including service principal credentials.

Here are the logs from the pod:

I0418 23:19:01.083364       1 utils.go:114] Using verbosity level 3 from environment variable APPGW_VERBOSITY_LEVEL
I0418 23:19:01.112049       1 supported_apiversion.go:70] server version is: 1.24.6
I0418 23:19:01.129428       1 environment.go:294] KUBERNETES_WATCHNAMESPACE is not set. Watching all available namespaces.
I0418 23:19:01.129464       1 main.go:118] Using User Agent Suffix='ingress-azure-fb89f64d8-5xdct' when communicating with ARM
I0418 23:19:01.129563       1 main.go:137] Application Gateway Details: Subscription="6398c0b0-8bc2-4fda-95f0-efb02e051900" Resource Group="driveb-dev" Name="driveb-application-gateway-dev"
I0418 23:19:01.129577       1 auth.go:58] Creating authorizer using Default Azure Credentials
I0418 23:19:01.129638       1 httpserver.go:57] Starting API Server on :8123
E0418 23:19:02.131543       1 authorizer.go:46] Error getting Azure token: DefaultAzureCredential: failed to acquire a token.
Attempted credentials:
        EnvironmentCredential: missing environment variable AZURE_TENANT_ID
        WorkloadIdentityCredential: missing environment variables for workload identity. Check webhook and pod configuration
        ManagedIdentityCredential: IMDS token request timed out
        AzureCLICredential: Azure CLI not found on path
E0418 23:19:02.218813       1 client.go:184] configuration error (bad request) or unauthorized error while performing a GET using the authorizer
E0418 23:19:02.218845       1 client.go:185] stopping GET retries
F0418 23:19:02.218916       1 main.go:175] Failed getting Application Gateway: Code="ErrorApplicationGatewayUnexpectedStatusCode" Message="Unexpected status code '401' while performing a GET on Application Gateway." InnerError="network.ApplicationGatewaysClient#Get: Failure responding to request: StatusCode=401 -- Original Error: autorest/azure: Service returned an error. Status=401 Code="AuthenticationFailedMissingToken" Message="Authentication failed. The 'Authorization' header is missing the access token.""

And, my helm config file looks like this:

# This file contains the essential configs for the ingress controller helm chart

# Verbosity level of the App Gateway Ingress Controller
verbosityLevel: 3

################################################################################
# Specify which application gateway the ingress controller will manage
#
appgw:
    applicationGatewayID: /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/driveb-dev/providers/Microsoft.Network/applicationGateways/driveb-application-gateway-dev
    usePrivateIP: false

    # Setting appgw.shared to "true" will create an AzureIngressProhibitedTarget CRD.
    # This prohibits AGIC from applying config for any host/path.
    # Use "kubectl get AzureIngressProhibitedTargets" to view and change this.
    shared: true

################################################################################
# Specify which kubernetes namespace the ingress controller will watch
# Default value is "default"
# Leaving this variable out or setting it to blank or empty string would
# result in Ingress Controller observing all acessible namespaces.
#
# kubernetes:
#   watchNamespace: <namespace>

################################################################################
# Specify the authentication with Azure Resource Manager
#
# Two authentication methods are available:
# - Option 1: AAD-Pod-Identity (https://github.com/Azure/aad-pod-identity)
armAuth:
    type: aadPodIdentity
    identityResourceID: /subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/MC_driveb-dev_driveb-aks-dev_eastus/providers/Microsoft.ManagedIdentity/userAssignedIdentities/app-gateway-identity
    identityClientID: 00000000-0000-0000-0000-000000000000

## Alternatively you can use Service Principal credentials
#armAuth:
#   type: servicePrincipal
#   secretJSON: 

################################################################################
# Specify if the cluster is RBAC enabled or not
rbac:
    enabled: true

[RobinFrcd]

Same issue here, just followed the official guide: https://learn.microsoft.com/en-us/azure/developer/terraform/create-k8s-cluster-with-aks-applicationgateway-ingress

[joshusrytls]

Same :\

[restenb]

Same issue here as well. No resolution?

[omeryesil]

Same here

[restenb]

The solution for me was to switch to workload identity https://learn.microsoft.com/en-us/azure/aks/workload-identity-overview?tabs=dotnet.

The documentation still seems incomplete but there's an AKS managed addon az aks update -g RESOURCE_GROUP -n CLUSTER_NAME --enable-workload-identity

[arpanD93]

I got it resolved after using "workload identity". As neither AAD Pod Identity (deprecated) nor Service Principal worked.

Below are the steps that I followed:

1. Register the ‘EnableWorkloadIdentityPreview’ feature flag

az feature register --namespace "Microsoft.ContainerService" --name "EnableWorkloadIdentityPreview"

2. Update an AKS cluster using the command with the --enable-oidc-issuer and the --enable-workload-identity parameter to use the OIDC Issuer and enable workload identity:

az aks update -g "${RESOURCE_GROUP}" -n myAKSCluster --enable-oidc-issuer --enable-workload-identity

3. Create a managed identity in the same resource group of AKS cluster.

az identity create --name "$MANAGEDIDNAME" --resource-group "$AKS_RG" --location "$LOCATION"

4. Provide the required roles to the managed identity:

• "read" - App GW Resource Group
• "contributor" - App GW
• "Network contributor" - VNet of App GW

5. Run the Helm install command (make sure to use armAuth type as workloadIdentity and the identityClientID of the Managed identity that was created in Step 3, in the helm-config.yaml file)

helm install $Installation_NAME -f helm-config.yaml application-gateway-kubernetes-ingress/ingress-azure -n $NAMESPACE

[Note: This command basically creates a SA in AKS which references the managed identity. At this point if you check the AGIC Pod, it should be in "crashedloopback" state]

6. Get the SA that got created

kubectl get sa -n $NAMESPACE

7. create federated id against that k8 SA

az identity federated-credential create --name $(Any Name) --identity-name $(Managed_Identity_Name) --resource-group $(Managed_Identity_Name_RG) --issuer "$OIDC_issuerURL" --subject system:serviceaccount:"$NAMESPACE":"$(The SA that got created after running HELM command)" --audience api://AzureADTokenExchange

8. Delete the AGIC Pod and let it re-create, it should work fine.