Open elouanKeryell-Even opened 7 months ago
We are observing the same random behavior with argocd: v2.5.18+b56ef05
msg="invalid client_secret on token request for client: argo-cd"
appears for each failed attempt in pod "argocd-dex-server" logs:
time="2024-02-21T15:13:54Z" level=info msg="username \"xxxxxxxxx\" mapped to entry uid=xxxxxxxx,ou=people,dc=xxxxxxxxx,dc=fr"
time="2024-02-21T15:13:54Z" level=info msg="performing ldap search ou=xxxxxxx,dc=xxxxxxxx,dc=fr sub (&(objectClass=groupOfNames)(member=uid=xxxxxxxx,ou=people,dc=xxxxxxxx,dc=fr))"
time="2024-02-21T15:13:54Z" level=info msg="login successful: connector \"ldap\", username=\"firstName lastName" \", preferred_username=\"\", email=\"xxxxxxxxx@xxxxxxxxx.fr\", groups=[\"xxxxxxxxxx\"]"
time="2024-02-21T15:13:54Z" level=info msg="invalid client_secret on token request for client: argo-cd"
Several attempts fail then suddenly one succeeds.
I'm seeing similar experience on v2.9.0+9cf0c69
Session 1
# dex server
time="2024-03-15T10:19:52Z" level=info msg="garbage collection run, delete auth requests=0, auth codes=4, device requests=0, device tokens=0"
time="2024-03-15T10:20:46Z" level=info msg="login successful: connector \"github\", username=\"USER_REDACTED\", preferred_username=\"USER_REDACTED\", email=\"USEREMAIL@REDACTED\", groups=[\"postman-eng:developers\"]"
time="2024-03-15T10:21:02Z" level=info msg="invalid client_secret on token request for client: argo-cd"
time="2024-03-15T10:21:08Z" level=info msg="login successful: connector \"github\", username=\"USER_REDACTED\", preferred_username=\"USER_REDACTED\", email=\"USEREMAIL@REDACTED\", groups=[\"postman-eng:developers\"]"
time="2024-03-15T10:21:18Z" level=info msg="login successful: connector \"github\", username=\"USER_REDACTED\", preferred_username=\"USER_REDACTED\", email=\"USEREMAIL@REDACTED\", groups=[\"postman-eng:developers\"]"
time="2024-03-15T10:21:30Z" level=info msg="login successful: connector \"github\", username=\"USER_REDACTED\", preferred_username=\"USER_REDACTED\", email=\"USEREMAIL@REDACTED\", groups=[\"postman-eng:developers\"]"
time="2024-03-15T10:21:33Z" level=info msg="login successful: connector \"github\", username=\"USER_REDACTED\", preferred_username=\"USER_REDACTED\", email=\"USEREMAIL@REDACTED\", groups=[\"postman-eng:developers\"]"
time="2024-03-15T10:21:43Z" level=info msg="login successful: connector \"github\", username=\"USER_REDACTED\", preferred_username=\"USER_REDACTED\", email=\"USEREMAIL@REDACTED\", groups=[\"postman-eng:developers\"]"
# argo server
time="2024-03-15T10:21:53Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=Get grpc.service=cluster.SettingsService grpc.start_time="2024-03-15T10:21:53Z" grpc.time_ms=1.113 span.kind=server system=grpc
time="2024-03-15T10:22:00Z" level=info msg="Performing authorization_code flow login: https://argo-mng.eu.postman-alpha.com/api/dex/auth?client_id=argo-cd&redirect_uri=https%3A%2F%2Fargo-mng.eu.postman-alpha.com%2Fauth%2Fcallback&response_type=code&scope=openid+profile+email+groups&state=REDACTED"
2024/03/15 10:22:03 http: proxy error: context canceled
# dex server
time="2024-03-15T10:21:58Z" level=error msg="Failed to authenticate: github: get teams: Get \"https://api.github.com/orgs/postman-eng/members/USER_REDACTED\": context canceled"
time="2024-03-15T10:22:00Z" level=error msg="Failed to authenticate: github: get teams: github: get URL Get \"https://api.github.com/user/teams\": context canceled"
time="2024-03-15T10:22:03Z" level=error msg="Failed to authenticate: github: get teams: github: get URL Get \"https://api.github.com/user/teams\": context canceled"
time="2024-03-15T10:22:15Z" level=info msg="login successful: connector \"github\", username=\"USER_REDACTED\", preferred_username=\"USER_REDACTED\", email=\"USEREMAIL@REDACTED\", groups=[\"postman-eng:developers\"]"
Session 2
# dex server
time="2024-03-15T12:19:09Z" level=info msg="login successful: connector \"github\", username=\"USER_REDACTED\", preferred_username=\"USER_REDACTED\", email=\"USEREMAIL@REDACTED\", groups=[\"postman-eng:developers\" \"postman-eng:kubernetes-engineers\" \"postman-eng:ci-cd-engineers\"]"
time="2024-03-15T12:19:30Z" level=info msg="Notifying 1 settings subscribers: [0x4000f7c840]"
time="2024-03-15T12:19:30Z" level=info msg="dex config unmodified"
time="2024-03-15T12:20:39Z" level=error msg="Failed to authenticate: github: get teams: github: get URL Get \"https://api.github.com/user/teams\": context canceled"
time="2024-03-15T12:20:41Z" level=info msg="login successful: connector \"github\", username=\"USER_REDACTED\", preferred_username=\"USER_REDACTED\", email=\"USEREMAIL@REDACTED\", groups=[\"postman-eng:developers\" \"postman-eng:kubernetes-engineers\" \"postman-eng:ci-cd-engineers\"]"
time="2024-03-15T12:20:51Z" level=error msg="Failed to authenticate: github: get teams: Get \"https://api.github.com/orgs/postman-eng/members/USER_REDACTED\": context canceled"
time="2024-03-15T12:20:54Z" level=info msg="login successful: connector \"github\", username=\"USER_REDACTED\", preferred_username=\"USER_REDACTED\", email=\"USEREMAIL@REDACTED\", groups=[\"postman-eng:developers\" \"postman-eng:kubernetes-engineers\" \"postman-eng:ci-cd-engineers\"]"
+1 for this behaviour with Okta SAML and version v2.10.7. Restarting the server deployment seemed to fix
we have noticed the same behaviour with v2.11.7. Randomly, it does not work and dex logs the error message:
invalid client_secret on token request
Checklist:
argocd version
.Describe the bug
Authentication through UI often fails with the following error:
(see screenshots further down)
sometimes, opening a new tab, re-typing argocd homepage URL and retrying the login process, makes it work
Our setup:
IMPORTANT: something we noticed is the errors seem to occur when interacting with the OLDEST INSTANCE
our two argocd-server instance:
dex config (in configmap
argocd-cm
):istio destination rule to setup sticky session for our multiple argocd-server instances:
To Reproduce
Error does not happen all the time, but here is my process:
Expected behavior
I expect to be able to successfully login
Screenshots
Version
Logs
Login failure:
Login success: