JUnit5 container does not work well with Arquillian transaction extension

bartoszmajsak commented 3 years ago

@tosix1988 commented on Wed Mar 10 2021

Hello,

we have been recently migrating out arquillian tests from JUnit4 container to JUnit5 container, and it mostly went without issues. However, we have spotted that the arquillian JTA extension has some issues with JUnit5 container.

Simply put, it seems that if the test is annotated with Transactional(value = TransactionMode.COMMIT), the test transaction is still rolled back (with JUnit4 this works as expected). I wasn't able to find much during debugging, but I have noticed the following:

in org.jboss.arquillian.transaction.impl.lifecycle.TransactionHandler, the testRequiresRollbackDueToFailure method always returns true, because testResultInstance.get() is always null. The TestContext is active at that point, but it stores no TestResult instance (even though the test passed). On JUnit4, the TestResult instance is present, and the testRequiresRollbackDueToFailure returns correct result.
with JUnit4 container, both ClientTestExecuter and ContainerTestExecuter are invoked; with JUnit 5, only ClientTestExecuter gets invoked (I am not sure if this makes any difference).

I don't know if this is the proper place to file this bug, or whether the arquillian-extension-transaction project would be a better place.

To ease investigation, I have attached two sample projects that should help with reproducing the issue. transactional-junit4.zip transactional-junit5.zip

OS details:

Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: C:\programs\apache-maven-3.6.3
Java version: 1.8.0_282, vendor: AdoptOpenJDK, runtime: C:\Program Files\Java\jdk-8.0.282\jre
Default locale: sk_SK, platform encoding: Cp1250
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

aminchegeni commented 1 year ago

Hi every one I'm happy to announce that after 2 days of debugging I finally found the problem.

The problem lies in 'insideArquillian' property when set to true in JUnitJupiterTestRunner class and it's execute method.

When I change this value manually in debug mode, the junit5 test cases pass.

Mr. @bartoszmajsak can you tell us how to change this property value and which side effect after change may be occurred?

aminchegeni commented 1 year ago

Result of running transactional-junit5.zip attached.

chengfang commented 1 year ago

Thanks @aminchegeni for debugging this. Will take a look.

aminchegeni commented 1 year ago

Hello Mr. @chengfang

If you need additional info, please tell us.

Best regards.

pemko123 commented 11 months ago

Thanks @aminchegeni for debugging this. Will take a look.

do you have any new infos?

TheOnlyAl commented 7 months ago

Hi there.

since i really believe this is a problem in arquillian-core i opened another ticket: https://github.com/arquillian/arquillian-core/issues/539

I also created a small project which provides a workaround for this problem: https://github.com/TheOnlyAl/arquillian-junit5-container-testresultfix

I tested my workaround against the transactional-junit5.zip from @tosix1988 and it seemed to work fine.

Maybe this helps with a fix in the future. :)

arquillian / arquillian-extension-transaction

JUnit5 container does not work well with Arquillian transaction extension #31