Bug Report: HTTP 403 Forbidden response for correctly authenticated and authorized calls to protected JAX-RS resource/FISH-7515

TheOnlyAl commented 1 year ago

Brief Summary

If you have a protected Web Resource and are Logged in using an authentication mechanism like @CustomFormAuthenticationMechanismDefinition, you can sometimes get a HTTP 403 Foprbidden status code if the server is under load.

Additional information

This is probably some kind of concurrency issue.

After further investigation it might be some problem with the @AutoApplySession functionality which uses the following interceptor: AutoApplySessionInterceptor.java

The problem seems to be that the actual additional information which is returned is not correct for a call under load: BaseContainerCallbackHandler.java

The SecurityContext.getCurrent().getAdditionalPrincipal() will return null for the calls which result in a 403 unauthorized. And since the group information seems to be part of the additional information this is a problem.

There seems to be an really old bug with a similar problem. https://github.com/payara/Payara/issues/751

Expected Outcome

As long as you are logged in correctly, you should not get a HTTP 403 Foprbidden.

Current Outcome

Sometimes calls will return with HTTP 403 Foprbidden while the server is under load.

Reproducer

I created a project with files to reproduce this: https://github.com/TheOnlyAl/payara-jaspic-bug

The current code is written for Java EE 8 and can be tested on Payara 5. If needed i could create a branch to check this for Jakarta EE 10 with Payara 6.

Use the project to build the jaspic-bug.war Web Archive.
Start a Payara Server using docker. For example: docker run -p 8080:8080 -p4848:4848 payara/server-full:5.2022.5.
Deploy the Application
Browse to http://localhost:8080/jaspic-bug/
Click on Login to authenticate
Click on Check Security. This will start a periodically check which will check if the protected resource can be accessed. This will write a + for every successfull call
To generate load you can use JMeter and use the provided test plan from /src/test/resources/CreateLoad.jmx. For example like this: ./jmeter -n -t src/test/resources/CreateLoad.jmx
When the Bug occurs you should get this output: Ending Security check with result status: 403

Operating System

Docker Image on Windows 10 using WSL 2

JDK Version

Checked for JDK 8 (Payara 5.2022.5) and JDK 17 (Payara 6.2021.1)

Payara Distribution

Payara Server Full Profile

TheOnlyAl commented 1 year ago

Hi.

Just wanted to know if you had success in replicating the problem or if i maybe can help in any way?

shub8968 commented 1 year ago

Hi @TheOnlyAl,

I have escalated this to our platform development team as FISH-7515.

payara / Payara