Azure Vault and Managed Permissions: Unable to get value using Managed identity

knobel-dk commented 6 months ago

Please provide us with the following information:

This issue is a: (mark with an x)

[x] bug report -> please search issues before submitting
[ ] documentation issue or request
[ ] regression (a behavior that used to work and stopped in a new release)

Issue description

I followed your guide here but I get an error, see below.

I specifically set it to system given this link.

See that it is enabled on the Container App:

And that the role is attached:

Steps to reproduce

Save the YAML below
Run az containerapp create -n $appName -g $resourceGroup --yaml [YAML_BELOW]

Expected behavior [What you expected to happen.] I expect the service to be created

Actual behavior [What actually happened.] ERROR: (InvalidParameterValueInContainerTemplate) The following field(s) are either invalid or missing. Field 'configuration.secrets' is invalid with details: 'Invalid value: "secretlicensekey": Unable to get value using Managed identity system for secret secretlicensekey. Error: managed Identity with resource Id system was not found when trying to get secret secretlicensekey from Azure Key Vault';.

Additional context

Using azure-cli 2.60.0

The YAML file:

name: zzzzzzzzzzzzzzzzz
type: Microsoft.App/containerApps
location: North Europe
properties:
  managedEnvironmentId: "/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/yyyyyyyyyyyyyyy/providers/Microsoft.App/managedEnvironments/zzzzzzzzzzzzzzzzz"
  environmentId: "/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/yyyyyyyyyyyyyyy/providers/Microsoft.App/managedEnvironments/zzzzzzzzzzzzzzzzz"
  workloadProfileName: Consumption
  configuration:
    activeRevisionsMode: Single
    ingress:
      external: true
      targetPort: 80
      transport: Auto
      traffic:
        - weight: 100
          latestRevision: true
      customDomains:
      allowInsecure: false
    secrets:

      - name: secretpassword
        value: ACR_PASSWORD

      - name: secretlicensekey
        keyVaultUrl: 'https://zzzzzzzzzzzzzzzzz.vault.azure.net/secrets/SECRET-LICENSE-KEY'
        identity: 'system'

    registries:
      - server: REGISTRYSERVER
        username: ACR_USERNAME
        passwordSecretRef: secretpassword

  template:
    revisionSuffix: UNIQUE_SUFFIX_TO_TRIGGER_UPDATE
    containers:
      - image: IMAGENAME
        name: CONTAINERNAME
        env:
          - name: SECRET_LICENSE_KEY
            secretRef: secretlicensekey

        resources:
          cpu: 0.25
          memory: 0.5Gi
        probes:
          - type: Liveness
            httpGet:
              path: "/"
              port: 80
              scheme: HTTP
            initialDelaySeconds: 60
            periodSeconds: 10
    initContainers:
    scale:
      minReplicas: 1
      maxReplicas: 2
      rules:
        - name: cpu-over-sixty
          custom:
            type: cpu
            metadata:
              type: Utilization
              value: "60"
    volumes: []
    serviceBinds:
identity:
  type: None

howang-ms commented 6 months ago

In the YAML you provided above, you didn't enable the system assigned managed identity. You should use something like this in your YAML:

identity:
  type: SystemAssigned

For more detail, please refer to here.

vturecek commented 6 months ago

This is the problem in the YAML spec:

identity:
  type: None

It should be:

identity:
  type: SystemAssigned

knobel-dk commented 6 months ago

I will check tomorrow. Thank you

knobel-dk commented 6 months ago

I did as adviced and am still getting ERROR: (InvalidParameterValueInContainerTemplate) The following field(s) are either invalid or missing. Field 'configuration.secrets' is invalid with details: 'Invalid value: "statamickey": Unable to get value using Managed identity system for secret statamickey. Error: unable to fetch secret 'statamickey' using Managed identity 'system'';.

Here is the output from az containerapp create -n $appName -g $resourceGroup --yaml [YAML_ABOVE]

{
  "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/YYYYYYYYYYYY/providers/Microsoft.App/containerapps/yyyyyyyyyyyyy",
  "identity": {
    "principalId": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "tenantId": "***",
    "type": "SystemAssigned"
  },
  "location": "North Europe",
  "name": "yyyyyyyyyyyyy",
  "properties": {
    "configuration": {
      "activeRevisionsMode": "Single",
      "dapr": null,
      "ingress": {
        "additionalPortMappings": null,
        "allowInsecure": false,
        "clientCertificateMode": null,
        "corsPolicy": null,
        "customDomains": null,
        "exposedPort": 0,
        "external": true,
        "fqdn": "yyyyyyyyyyyyy.somename-804dbc47.northeurope.azurecontainerapps.io",
        "ipSecurityRestrictions": null,
        "stickySessions": null,
        "targetPort": 80,
        "traffic": [
          {
            "latestRevision": true,
            "weight": 100
          }
        ],
        "transport": "Auto"
      },
      "maxInactiveRevisions": 100,
      "registries": [
        {
          "identity": "",
          "passwordSecretRef": "ZZZZZZZZZZZZZZZZZZZZZZZZZ",
          "server": "zzzzzzzzzzzzzzzz.azurecr.io",
          "username": "zzzzzzzzzzzzzzzz"
        }
      ],
      "secrets": [
        {
          "name": "secretpassword"
        },
        {
          "name": "ZZZZZZZZZZZZZZZZZZZZZZZZZ"
        }
      ],
      "service": null
    },
    "customDomainVerificationId": "abcdabcdabcdabcdabcd",
    "delegatedIdentities": [],
    "environmentId": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/YYYYYYYYYYYY/providers/Microsoft.App/managedEnvironments/container-apps-env-yyyyyyyyyyyyy",
    "eventStreamEndpoint": "https://northeurope.azurecontainerapps.dev/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/YYYYYYYYYYYY/containerApps/yyyyyyyyyyyyy/eventstream",
    "latestReadyRevisionName": "yyyyyyyyyyyyy--9156474657",
    "latestRevisionFqdn": "yyyyyyyyyyyyy--9156474657.somename-804dbc47.northeurope.azurecontainerapps.io",
    "latestRevisionName": "yyyyyyyyyyyyy--9156474657",
    "managedEnvironmentId": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/YYYYYYYYYYYY/providers/Microsoft.App/managedEnvironments/container-apps-env-yyyyyyyyyyyyy",
    "outboundIpAddresses": [
      "SOME.IP.xxx.yyy",
      "SOME.IP.xxx.yyy",
      "SOME.IP.xxx.yyy",
      "...."
    ],
    "provisioningState": "Succeeded",
    "runningStatus": "Running",
    "template": {
      "containers": [
        {
          "image": "zzzzzzzzzzzzzzzz.azurecr.io/yyyyyyyyyyyyy:latest",
          "name": "yyyyyyyyyyyyy",
          "probes": [
            {
              "httpGet": {
                "path": "/",
                "port": 80,
                "scheme": "HTTP"
              },
              "initialDelaySeconds": 60,
              "periodSeconds": 10,
              "type": "Liveness"
            }
          ],
          "resources": {
            "cpu": 0.25,
            "ephemeralStorage": "1Gi",
            "memory": "0.5Gi"
          }
        }
      ],
      "initContainers": null,
      "revisionSuffix": "",
      "scale": {
        "maxReplicas": 2,
        "minReplicas": 1,
        "rules": [
          {
            "custom": {
              "metadata": {
                "type": "Utilization",
                "value": "60"
              },
              "type": "cpu"
            },
            "name": "cpu-over-sixty"
          }
        ]
      },
      "serviceBinds": null,
      "terminationGracePeriodSeconds": null,
      "volumes": []
    },
    "workloadProfileName": "Consumption"
  },
  "resourceGroup": "docs-datapult-dk",
  "systemData": {
    "createdAt": "2024-05-20T07:11:21.0834188",
    "createdBy": "***",
    "createdByType": "Application",
    "lastModifiedAt": "2024-05-20T12:18:45.7886352",
    "lastModifiedBy": "***",
    "lastModifiedByType": "Application"
  },
  "type": "Microsoft.App/containerApps"
}

howang-ms commented 5 months ago

Hi @knobel-dk From the error message, it seems the system assigned identity don't have the permission to your key vault. Please note, you might need multiple steps to do this, since the system assigned identity is created with container app. You need to create a dummy app first to get the object id the system assigned identity.

rchauvetclara commented 1 month ago

Hello, i have the same issue here. I try to add secret with a user assign managed identity. `Failed to update secrets: Failed to provision revision for container app 'xxxxxxxxxxxx'. Error details: The following field(s) are either invalid or missing. Field 'configuration.secrets' is invalid with details: 'Invalid value: "xxxxxxxxxx": Unable to get value using Managed identity xxxxxxxxxxxx for secret xxxxxxxxx. Error: unable to fetch secret 'xxxxxxxxxx' using Managed identity

is there a work around ?

leartbeqiraj0 commented 1 week ago

@rchauvetclara this is happening in at least one of two use cases: 1) Container App Identity does not have permissions to pull from the key vault. 2) Trying to reference a non existing secret, a misspelled name, an incorrect identifier, or a generally malformed Key Vault URI.

Now I can't see/edit the other secrets now the only way out is to delete and recreate the container app.

Did you find any solution yet?

cf-rdegregory commented 1 day ago

A somewhat similar issue here is that when using a System Assigned Identity to access a Key Vault Secret that contains the registry pull credentials, the Container App (or Container App Job) perpetually hangs during provisioning, as a circular dependency is introduced: the Key Vault Secrets Role can't be assigned to the Managed Identity until the resource is deployed, but the resource can't complete its deployment because it can't pull the image.

With regards to this recommendation:

Please note, you might need multiple steps to do this, since the system assigned identity is created with container app. You need to create a dummy app first to get the object id the system assigned identity.

This really doesn't make any sense, as the dummy app will have a different Object ID for its System Assigned identity than the target app...

My only workaround has been to use a separate User-assigned identity for each Container App / Job in the Environment, with the Key Vault Secrets Role assigned prior to the creation of the ACA resource. This is cumbersome and results in a lot of unnecessary extra UMIs created as part of our solution.

@howang-ms @anthonychu any advice here?

EDIT: #836 details my specific issue.

microsoft / azure-container-apps