bug: confluence package gets sso error when re-deployed on existing deployment - fails validation

[JoeHCQ1]

In testing #193 I found that when I re-deployed confluence over a working install the validation step failed. Looking further, the pod was just fine, it was the UDS package which had bad events from the UDS operator.

Here is what I got from asking kubectl to describe it:

Status:
  Authservice Clients:
  Endpoints:
    confluence.mtsi-dev.bigbang.dev
  Monitors:
    confluence-metrics
  Network Policy Count:  11
  Observed Generation:   2
  Phase:                 Ready
  Retry Attempt:         0
  Sso Clients:
    uds-package-confluence-saml
Events:
  Type     Reason                Age    From              Message
  ----     ------                ----   ----              -------
  Warning  ReconciliationFailed  8m21s  uds.dev/operator  Waiting 3 seconds before retrying package
  Warning  ReconciliationFailed  6m19s  uds.dev/operator  Failed to process Keycloak request for client 'uds-package-confluence-saml', package confluence/confluence. Error: "Unauthorized", {"error":"invalid_token","error_description":"Not authorized to update client. Maybe missing token or bad token type."}
  Warning  ReconciliationFailed  8m8s   uds.dev/operator  Waiting 27 seconds before retrying package
  Warning  ReconciliationFailed  7m41s  uds.dev/operator  Waiting 81 seconds before retrying package
  Warning  ReconciliationFailed  8m21s  uds.dev/operator  Failed to set token in store for client 'uds-package-confluence-saml', package confluence/confluence
  Warning  ReconciliationFailed  8m17s  uds.dev/operator  Waiting 9 seconds before retrying package
  Warning  ReconciliationFailed  8m8s   uds.dev/operator  Failed to process Keycloak request for client 'uds-package-confluence-saml', package confluence/confluence. Error: "Unauthorized", {"error":"invalid_token","error_description":"Not authorized to update client. Maybe missing token or bad token type."}
  Warning  ReconciliationFailed  7m41s  uds.dev/operator  Failed to process Keycloak request for client 'uds-package-confluence-saml', package confluence/confluence. Error: "Unauthorized", {"error":"invalid_token","error_description":"Not authorized to update client. Maybe missing token or bad token type."}
  Warning  ReconciliationFailed  8m17s  uds.dev/operator  Failed to process Keycloak request for client 'uds-package-confluence-saml', package confluence/confluence. Error: "Unauthorized", {"error":"invalid_token","error_description":"Not authorized to update client. Maybe missing token or bad token type."}

This left the package in a failed state, not READY, which fails the validation step.

[JoeHCQ1]

See @jacobbmay with similar issue here: https://defense-unicorns.slack.com/archives/C06QJAUHWFN/p1727361665054139

[JoeHCQ1]

There's no action for us here. It's related to deleting and then redeploying apps, which is a dev-focused task, not customer facing. It's not a clear enough error to reproduce or important enough that we're reporting it to product, so it's a null action.