current_user helper returning nil after successful authentication.

Environment

Ruby 3.1.4
Rails 7.1.1
Devise 4.9.3
Nginx 1.20.1 on Passenger 6.0.18

Current behavior

I bumped my application to Rails 7.1, and on my development server, signing in using my Devise setup continues to work fine. However, on my staging server (RHEL7 using nginx), authentication no longer works.

Here are the clues I have gathered after two days straight of debugging:

At first, it claims that it cannot verify the authenticity_token. The token is confirmed being provided in the as well as a hidden field in the sign-in form. I added skip_forgery_protection in my locally-provided Devise::SessionsController (with no other modifications from the file generated by the gem) just to get it working. Weirdly, removing protect_from_forgery from my ApplicationController entirely, as well as removing both authenticity_token tags, did not stop the CSRF error during sign-in). For what it's worth, I did apply to protect_from_forgery prepend: true as the wiki suggests, and nothing changed. Including by removing it all together. I'm not sure if this is a clue or a red herring.

Once I stopped seeing the CSRF error in the logs, I had a different problem. I authenticated, which would redirect me to a page that requires authentication, then that page would redirect me back to sign-in. In the logs, I see Devise increment my user record's log_in_count, and within the session#create action I could log the authenticated user object, so the authentication was accepted. But by the next page load, it would act like I'm not logged in, with a nil current_user on any page and redirect to sign_in page via before_action :authenticate_user! So my hunch became that the current_user value was not being properly set in the session cookie, so I started messing with that. I was able to recreate this symptom on my development server if I set my cookie_store config to use secure: true on development (previously it was only set to be secure on non-dev envs). However, switching secure: false didn't help staging at all.

Also worth noting that signing out behaves similarly, it redirects to the after_sign_out_path_for page, but the user is never signed out, implying it never actually changes the authenticated user data.

So, what my problem is not:

Turbo interaction (form submits successfully)
Namespace collision or other major codebase issue (behaved properly before Rails 7.1 upgrade and continues to work correctly on development)

What it feels like to me:

Something regarding reading/setting the session cookie during the login/logout process
An adverse interaction with a new Rails 7.1 config change, but I can't for the life of me find anything that seems relevant to accessing cookies.

Expected behavior

Able to access authenticated user successfully.

Any thoughts about whether this could be related to recent changes to Devise or Rails, or areas to investigate beyond those mentioned above would be greatly appreciated. It would be hard to share a sample repo with this bug but I can certainly share my current code on request.

heartcombo / devise