pegasystems / pega-helm-charts

Orchestrate a Pega Platform™ deployment by using Docker, Kubernetes, and Helm to take advantage of Pega Platform Cloud Choice flexibility.
https://community.pega.com/knowledgebase/articles/cloud-choice
Apache License 2.0
120 stars 194 forks source link

AKS Workload Identity to connect Azure SQL DB with JDBC #772

Open ursusca opened 1 week ago

ursusca commented 1 week ago

Describe the bug Microsoft replaces pod-managed identity (preview) authentication method, which described in the documentation, with Microsoft Entra Workload ID. We are trying to switch from aadpodidbinding to workload identities but it's not working

To Reproduce Modify pega.yaml file for helm installation:

      # Managed Identities
      podLabels:
        azure.workload.identity/use: "true"

      custom:
        serviceAccountName: sa-sandbox-dev

Create service account:

kubectl apply -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa-sandbox-dev
  namespace: sandbox-dev
  annotations:
    azure.workload.identity/client-id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    azure.workload.identity/tenant-id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
  labels:
    azure.workload.identity/use: "true"
EOF

Deploy pega platform using helm. After deployment we can see, that settings were applied to pods:

  labels:
    app: pega-web
    azure.workload.identity/use: "true"
...
  serviceAccount: sa-sandbox-dev
  serviceAccountName: sa-sandbox-dev

Also, a pod has environment variables set and token file created:

pegauser@pega-web-674c5b6b8-lrs9f:/$ env | grep -i azure
AZURE_TENANT_ID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
AZURE_FEDERATED_TOKEN_FILE=/var/run/secrets/azure/tokens/azure-identity-token
AZURE_AUTHORITY_HOST=https://login.microsoftonline.com/
AZURE_CLIENT_ID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

In DEV STUDIO: Records -->SysAdmin --> Database --> Create new connection using JDBC URL

jdbc:sqlserver://xxxxxxx.xxxxxxxx.database.windows.net:1433;database=xxxxxxxx;selectMethod=cursor;sendStringParametersAsUnicode=false;authentication=ActiveDirectoryMSI;msiClientId=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx;encrypt=true;trustServerCertificate=true

Then click Test connection and got the error:

Status Value:
Database-General Connection Exception 0 HikariPool-3 - Connection is not available, request timed out after 10000ms. DatabaseException caused by prior exception: java.sql.SQLTransientConnectionException: HikariPool-3 - Connection is not available, request timed out after 10000ms. | SQL Code: 0 | SQL State: null DatabaseException caused by prior exception: com.microsoft.sqlserver.jdbc.SQLServerException: Failed to load MSAL4J Java library for performing ActiveDirectoryDefault authentication. | SQL Code: 0 | SQL State: null

Next, we tried to import msal4j-1.15.1.jar file and we still got the same error. Next, we tried to create custom image to include jar files:

>cat Dockerfile
FROM pega-docker.downloads.pega.com/platform/pega:8.23.1

COPY *.jar /usr/local/tomcat/lib/

>ls -l
azure-core-1.49.1.jar
azure-core-http-netty-1.15.1.jar
azure-core-test-1.26.0.jar
azure-identity-1.12.2.jar
azure-json-1.1.0.jar
azure-security-keyvault-keys-4.8.4.jar
azure-xml-1.0.0.jar
jackson-annotations-2.17.1.jar
jackson-core-2.17.1.jar
jackson-databind-2.17.1.jar
jackson-datatype-jsr310-2.17.1.jar
jna-platform-5.14.0.jar
json-smart-2.5.1.jar
msal4j-1.15.1.jar
msal4j-persistence-extension-1.3.0.jar
oauth2-oidc-sdk-11.12.jar
reactor-core-3.6.7.jar
slf4j-api-2.0.13.jar

And re-deploy pega platform. Test Connection throws an error:

There has been an issue; please consult your system administrator

And in the pod logs:

Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.azure.identity.ManagedIdentityCredentialBuilder

Expected behavior Expecting successful connection

Chart version Tried 3.23.0 and 3.17.1

Server (if applicable, please complete the following information):

misterdorito commented 1 week ago

This is a topic we might not be able to help you with, but there is one other thing I would suggest you try.

You had mentioned that you had imported msal4j-1.15.1.jar -- presumably you mean via the Import Wizard in the web portal. You should try your custom image against a newly installed pega environment (or delete the imported classes from the database). Having the jar in the database and on the filesystem can lead to some unhappy classloading behavior.

Also, if all that you are doing with the custom image is to load jar files to the tomcat lib dir, you specify URLs for each jar file delimited by semicolons and they will be downloaded and copied there.