support for Kubernetes Event-driven Autoscaling workload identity

JonasCordsen commented 1 year ago

Describe the solution you'd like it would be great to support use of workload identity (With a managed identity) in KEDA in order to handle the secrets that is used by trigger

IE guide form keda https://keda.sh/docs/2.9/authentication-providers/azure-key-vault/

Describe alternatives you've considered I have considered to setup KEDA by hand, but the support for this would make it a lot easier

EppO commented 1 year ago

KEDA 2.10 add-on deployed within AKS 1.26 supports workload identity but it requires extra setup

Create a Managed Identity with the corresponding permissions based on the external azure resources you want to access (Keyvault, Azure Monitor Workspace, ...) and save its Client ID for later
Get OIDC issuer URL of your cluster: export AKS_OIDC_ISSUER="$(az aks show -n "<AKS cluster name>" -g "<RGP name>" --query "oidcIssuerProfile.issuerUrl" -otsv)"
Create federated credentials: az identity federated-credential create --name "<Federated Credential name>" --identity-name "<Managed Id name>" --resource-group "<RGP name>" --issuer "${AKS_OIDC_ISSUER}" --subject system:serviceaccount:kube-system:keda-operator --audience api://AzureADTokenExchange

~~Add Managed Identity client ID config in the keda-operator's service account in kube-system namespace:~~ NOT NEEDED per @kratkyzobak's comment

apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
annotations:
azure.workload.identity/client-id: <Managed ID Client ID>

~~Restart the keda-metrics-apiserver pods~~
~~keda-metrics-apiserver pods should have an environment variable AZURE_CLIENT_ID set with the value you set in the service account~~

Add the User Managed Identity Client ID to the

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: trigger-authentication-name
namespace: default # must be same namespace as the ScaledObject
spec:
podIdentity:
provider: azure-workload
identityId: <Managed ID Client ID>

I'm honestly not sure about overriding the service account manifest with the Managed Id's client ID, not sure it's future proof in terms of AKS upgrades. I would be nice if the AKS add-on provisioning was taking care about the Managed Id provisioning + federated credentials for us, or at least having CLI option to give the client ID ourselves, or even better support for both solutions.

kratkyzobak commented 1 year ago

azure.workload.identity/client-id annotation should not be required to modify. You should specify clientId in podIdentity.identityId for each specific TriggerAuthentication resource.

EppO commented 1 year ago

that's awesome! no need to do this ugly hack then, thanks for the tip!! I edited my original comment for the updated recipe