Open gmsoft-tuxicoman opened 10 months ago
One way to investigate is to configure pomerium to use Postgres databroker backend.
Then look for the contents of pomerium.records table that have "Session" in type column.
Next look for any errors in pomerium logs from identity_manager. There's a background process that continuously refreshes sessions with the IdP
I've done that but it doesn't provide more info than what I already had and provided above.
The request to the IdP is failing with this:
The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed. Client credentials missing or malformed in both HTTP Authorization header and HTTP POST body.
Unfortunately I can't see what requests pomerium is issuing to Jumpcloud. How can I troubleshoot that ?
@gmsoft-tuxicoman
A few quick thoughts looking at your config:
Otherwise, If you are able to build from source and add additional log messages to the refresh call would be the best way to troubleshoot why the interaction between Pomerium's session refresh & jumpcloud is failing conclusively.
@desimone I deleted my previous post. It seems that the issue keeps occurring. For some reason the refresh session query provides empty client id/secret when refreshing the session token.
I used mitmproxy to sniff the traffic between pomerium and Jumpcloud. Pomerium send Authorization: Basic Og==
. This is just :
for the credential.
Shouldn't pomerium provide some kind of authentication ?
The only 2 parameters in the query are : grant_type: refresh_token
, refresh_token: <old refresh_token>
.
Ok it looks like I was reviewing old logs. Adding the client id and secret in the top does solve the issue. When I review the queries made by pomerium for the refresh_token, I definitely see the client_id and client_secret now.
Anything I can do for this one ? Thanks
@gmsoft-tuxicoman, is using a single global IdP client ID & secret a viable workaround for your deployment?
I think the underlying issue here is that session refresh will use only the global IdP settings, rather than any per-route IdP client ID / secret.
Session refresh is handled by a component called the identity manager. It looks like the identity manager is configured with a single IdP authenticator, which is created based on the main IdP configuration settings.
@kenjenkins Yes I have configured a single idp entry for now and doing the authorization in pomerium. That workaround is working.
What happened?
We've used Jumpcloud as an identity provider. We can authenticate correctly and see all the claims in verify. We are currently trying to provide access to jira and splunk. Both of those fail after a while. Looking at the HTTP logs, after a while, about 30 minutes, the API calls are redirected to the authenticate service URL to login again.
What did you expect to happen?
The session should be alive for more than 30 minutes.
How'd it happen?
Browse splunk.mycompany.com. Use splunk for a while. After ~30 mins API calls to splunk are redirected to the authenticate URL. The user facing error message is "Disconnected from Splunk server."
What's your environment like?
What's your config.yaml?
At first I tried without the idp_request_params but the problem is the same.
What did you see in the logs?
Additional context
I couldn't find a way to get logs from the OIDC queries made to the IdP. That would help me understand the issue.