Session resetting every request

[lorenzoaiello]

I've got an issue I've been chasing down for the last few days and I suspect that it's a configuration issue on my side, but I can't figure out what it is. Any help would be appreciated.

Expected Behavior

Users should be able to log in once through the OIDC user flow and maintain that session until the token expires.

Current Behavior

Accessing a webpage runs the user through the OIDC workflow successfully, but all static assets it tries to access redirect to restart the authentication workflow. Navigating to a new page also re-runs the authentication workflow.

The request FROM /callback after authentication has the correct Set-Cookie payload and it is being passed to the subsequent request.

The subsequent request to /queryview/ after authentication (4th line down) has the correct Cookie header, but gets a new session cookie returned.

If you look at the logs towards the bottom, you can see that something is happening to the session between requests.

Infrastructure

In case it's relevant, this is running on three EC2 instances in AWS behind an NLB which is doing SSL termination (re-establishing SSL to Kong).

Troubleshooting Attempted

I've already tried multiple things, including:

• Turning off the session checks in https://github.com/bungle/lua-resty-session as described in https://github.com/bungle/lua-resty-session/issues/23#issuecomment-172180840
• Set a session secret in Kong AND the OIDC plugin
• Set the session secret to blank in Kong and set it only in the OIDC plugin

Kong Configuration File

# Not actual secret, but the secret is exactly 32 characters, alpha-numeric string
nginx_proxy_set = $session_secret "xxx"

# Disabling headers
headers =

# Custom Plugins
plugins = bundled, oidc

# Database settings
database = postgres
pg_host = $DB_HOST
pg_user = $DB_USER
pg_password = $DB_PASSWORD
pg_database = $DB_NAME
pg_ssl = on

# Load balancer headers
real_ip_header = X-Forwarded-For
trusted_ips = 0.0.0.0/0

proxy_listen = 0.0.0.0:8443 ssl
admin_listen = 0.0.0.0:8444 ssl

OIDC Plugin Configuration

{
    "client_id": "****",
    "client_secret": "****",
    "discovery": "https://...mydomain.../.well-known/openid-configuration",
    "scope": "openid email profile",
    "introspection_endpoint": "https://...mydomain.../v1/introspect",
    "redirect_uri_path": "/callback"
}

Log Files

Jan 16, 2020 @ 07:00:24.829 | 2020/01/16 15:00:24 [debug] 30965#0: *144801 [lua] openidc.lua:1228: authenticate(): Authentication is required - Redirecting to OP Authorization endpoint

Jan 16, 2020 @ 07:00:24.829 | 2020/01/16 15:00:24 [debug] 30965#0: *144801 [lua] openidc.lua:1203: authenticate(): session.present=true, session.data.id_token=false, session.data.authenticated=nil, opts.force_reauthorize=nil, opts.renew_access_token_on_expiry=nil, try_to_renew=true, token_expired=false

Jan 16, 2020 @ 07:00:24.829 | 2020/01/16 15:00:23 [debug] 30965#0: *144801 [lua] handler.lua:46: make_oidc(): OidcHandler calling authenticate, requested path: /admin/queryview/

Jan 16, 2020 @ 07:00:25.830 | 2020/01/16 15:00:25 [debug] 30965#0: *144801 [lua] handler.lua:46: make_oidc(): OidcHandler calling authenticate, requested path: /callback?code=****&state=****

Jan 16, 2020 @ 07:00:25.831 | 2020/01/16 15:00:25 [debug] 30965#0: *144801 [lua] openidc.lua:1169: authenticate(): Redirect URI path (/callback) is currently navigated -> Processing authorization response coming from OP

Jan 16, 2020 @ 07:00:25.831 | 2020/01/16 15:00:25 [debug] 30965#0: *144801 [lua] openidc.lua:918: authenticate(): Authentication with OP done -> Calling OP Token Endpoint to obtain tokens

Jan 16, 2020 @ 07:00:26.831 | 2020/01/16 15:00:26 [debug] 30965#0: *144801 [lua] openidc.lua:978: authenticate(): OIDC Authorization Code Flow completed -> Redirecting to original URL (/admin/queryview/)

Jan 16, 2020 @ 07:00:26.832 | 2020/01/16 15:00:26 [debug] 30965#0: *144801 [lua] openidc.lua:1244: authenticate(): id_token={"ver":1,"amr":["swk","mfa","pwd"],"iat":1579186831,"iss":"****","name":"****","idp":"****","email":"****","aud":"****","jti":"****","exp":1579190431,"sub":"****","preferred_username":"****","auth_time":1579176922,"nonce":"****","at_hash":"****"}

Jan 16, 2020 @ 07:00:26.832 | 2020/01/16 15:00:26 [debug] 30965#0: *144801 [lua] openidc.lua:1203: authenticate(): session.present=true, session.data.id_token=true, session.data.authenticated=true, opts.force_reauthorize=nil, opts.renew_access_token_on_expiry=nil, try_to_renew=true, token_expired=false

Jan 16, 2020 @ 07:00:26.832 | 2020/01/16 15:00:26 [debug] 30965#0: *144801 [lua] handler.lua:46: make_oidc(): OidcHandler calling authenticate, requested path: /admin/queryview/

Jan 16, 2020 @ 07:00:27.832 | 2020/01/16 15:00:26 [debug] 30965#0: *144801 [lua] handler.lua:46: make_oidc(): OidcHandler calling authenticate, requested path: /admin/bootstrap/bootstrap3/swatch/default/bootstrap.min.css?v=3.3.5

Jan 16, 2020 @ 07:00:27.833 | 2020/01/16 15:00:26 [debug] 30965#0: *144877 [lua] handler.lua:46: make_oidc(): OidcHandler calling authenticate, requested path: /admin/bootstrap/bootstrap3/css/bootstrap-theme.min.css?v=3.3.5

Jan 16, 2020 @ 07:00:27.834 | 2020/01/16 15:00:27 [debug] 30965#0: *144877 [lua] openidc.lua:1203: authenticate(): session.present=nil, session.data.id_token=false, session.data.authenticated=nil, opts.force_reauthorize=nil, opts.renew_access_token_on_expiry=nil, try_to_renew=true, token_expired=false

[mamacdon]

I've been having a similar issue (although I am using shm storage). What finally fixed it for me was using a single nginx worker:

worker_processes  1;

Another workaround explained here also worked (this one lets you use any number of workers):

http {
  init_by_lua_block {
    require('resty.session')
  }
}

I'd previously tried the workarounds from here: disabling checks, setting a session secret and disabling encryption (set $session_cipher none), but they did not work.

Speculation: it seems like there's some initialization work that has to happen in the main nginx process before the workers start up. If you defer loading of resty.session until your access_by_lua phase, it creates some kind of race between the workers (even when they should all be using identical configuration?).

[ahmednasir91]

I had this issue and was resolved by adding session secret on kong KONG_NGINX_PROXY_SET: "$$session_secret <secret>"

[sumeshsg]

@ ahmednasir91, thanks, it's working

[mateodelnorte]

I'm also getting this error with the redis session provider.

Any solutions?

[liweitianux]

Hi @mamacdon, I think the issue is caused by that every worker process generates its own different session secret, and thus cannot decrypt the session cookie created by another worker process.

Have a look at https://github.com/bungle/lua-resty-session/issues/23#issuecomment-1223717732 and hope it could help you.