Closed yonirab closed 1 year ago
It looks like the upgrade to v3.4.8 was somehow causing all the data in my workflow-controller-configmap to get wiped!!!
And hence there was no issuer
found for my SSO configuration.
I saw this happen on 2 attempts to upgrade from 3.4.7 to 3.4.8. Both times all the data in my workflow-controller-configmap got wiped!!!
After reconfiguring the workflow-controller-configmap, the 3.4.8 argo-server finally agreed to start up without errors.
Does this make any sense??? Why on earth would upgrade from 3.4.7 to 3.4.8 cause the data in my workflow-controller-configmap to get wiped? Is there something I need to do to prevent that?
For context there's a slack discussion here: https://cloud-native.slack.com/archives/C01QW9QSSSK/p1686479990177909
I had a similar experience, but the culprit (at least for my case) was due to this setting: https://github.com/argoproj/argo-helm/blob/main/charts/argo-workflows/values.yaml#L628 . I misinterpreted this comment: "## SSO is activated by adding --auth-mode=sso to the server command line.".
For the longest time, I've had server.sso.enabled: false
because I've had the extraArgs configured. Since the latest release,
chart will not render the sso settings for the configmap if you do not have `server.sso.enabled: true." After updating this reference, everything worked OK.
I tried upgrading a different cluster to 3.4.8 and this time it worked fine. So not sure what the problem was in the first cluster, but I guess it must be something cluster specific, so this issue can be closed as far as I'm concerned.
Pre-requisites
:latest
What happened/what you expected to happen?
We have been running Argo Workflows on GKE for several years. On upgrade from 3.4.7 to 3.4.8 the argo-server is crashing with
Error: issuer empty
We run our argo-server with the following args and env:
Version
v3.4.8
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
Logs from the workflow controller
Logs from in your workflow's wait container