search

arquillian / arquillian-core

Arquillian provides a component model for integration tests, which includes dependency injection and container life cycle management. Instead of managing a runtime in your test, Arquillian brings your test to the runtime.
http://arquillian.org
Apache License 2.0
366 stars 196 forks source link

TestResult instance is not properly produced using Junit5 inside the container #539

Open TheOnlyAl opened 4 months ago

TheOnlyAl commented 4 months ago
Issue Overview

I believe there is a problem when running Junit5 tests in the container because the TestResult instance is not set before the After phase is executed. In my opionion this leads for example to the following issue: https://github.com/arquillian/arquillian-extension-transaction/issues/31

Junit4 seems to corretly inject the result. Propably by using UpdateTestResultBeforeAfter.java

Expected Behaviour

The TestResult should be set before the After phase is executed for Junit5 tests.

Current Behaviour

The TestResult is not set.

Steps To Reproduce

I created a small project (arquillian-junit5-container-testresultfix) which uses an custom Extension to check if the TestResult is set and also provides an Extension which provides a workaround for the problem.

To check if the result is not set, you could:

  1. Check out the project
  2. Run the Tests

The tests will all succeed as long the the result is not set. When this is fixed at least the UnfixedTestResultIT.java should fail

Additional Information

I did a lot of debugging for this problem since it basically makes most of my tests using transactions fail. It is quite hard to really understand where the problem is. It could be a missing interceptor, missing observer, or something else.

Since there is also seems currently no other easys way for extensions to get the Result of a test after its run but before the After phase is called, i am not sure how else extensions should work.

Checked for: Arquillian 1.8.0.Final

Please just tell me if you need any more information. I am happy to try to help as much as possible.

starksm64 commented 4 months ago

If I run the tests on the https://github.com/TheOnlyAl/arquillian-junit5-container-testresultfix.git project, they just hang with messages about java.lang.IllegalStateException: Unknown protocol: RFB

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running de.adtelligence.arquillian.junit5.container.testresultfix.tests.embedded.TestResultIT
Feb 20, 2024 6:15:12 AM com.sun.enterprise.v3.server.CommonClassLoaderServiceImpl findH2Client
INFO: Cannot find h2db client jar file, h2 jdbc driver will not be available by default.
Feb 20, 2024 6:15:12 AM org.hibernate.validator.internal.util.Version <clinit>
INFO: HV000001: Hibernate Validator null
Feb 20, 2024 6:15:13 AM com.hazelcast.instance.AddressPicker
INFO: [LOCAL] [development] [4.2.5] Using bind address: 0.0.0.0/0.0.0.0:4900
Feb 20, 2024 6:15:13 AM com.hazelcast.instance.AddressPicker
INFO: [LOCAL] [development] [4.2.5] Using public address: /fe80:0:0:0:f48e:dbff:fe18:70e9%anpi0:4900
Feb 20, 2024 6:15:13 AM com.hazelcast.system
INFO: [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 [development] [4.2.5] Hazelcast 4.2.5 (20220427 - 1dbec0d) starting at [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900
Feb 20, 2024 6:15:14 AM com.hazelcast.instance.impl.Node
INFO: [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 [development] [4.2.5] Using Discovery SPI
Feb 20, 2024 6:15:14 AM com.hazelcast.internal.diagnostics.Diagnostics
INFO: [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 [development] [4.2.5] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
Feb 20, 2024 6:15:14 AM com.hazelcast.core.LifecycleService
INFO: [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 [development] [4.2.5] [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 is STARTING
Feb 20, 2024 6:15:14 AM com.hazelcast.internal.server.tcp.TcpServerConnection
WARNING: [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 [development] [4.2.5] Connection[id=1, /fe80:0:0:0:f48e:dbff:fe18:70e9%6:56827->/fe80:0:0:0:f48e:dbff:fe18:70e9%anpi0:5900, qualifier=null, endpoint=[fe80:0:0:0:f48e:dbff:fe18:70e9]:5900, alive=false, connectionType=NONE, planeIndex=-1] closed. Reason: Exception in Connection[id=1, /fe80:0:0:0:f48e:dbff:fe18:70e9%6:56827->/fe80:0:0:0:f48e:dbff:fe18:70e9%anpi0:5900, qualifier=null, endpoint=[fe80:0:0:0:f48e:dbff:fe18:70e9]:5900, alive=true, connectionType=NONE, planeIndex=-1], thread=hz.compassionate_bell.IO.thread-in-0
java.lang.IllegalStateException: Unknown protocol: RFB
    at com.hazelcast.internal.server.tcp.UnifiedProtocolDecoder.onRead(UnifiedProtocolDecoder.java:117)
    at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:138)
    at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:383)
    at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:368)
    at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:294)
    at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249)
    at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)

Feb 20, 2024 6:15:14 AM com.hazelcast.internal.server.tcp.TcpServerConnection
WARNING: [fe80:0:0:0:f48e:dbff:fe18:70e9]:4900 [development] [4.2.5] Connection[id=2, /fe80:0:0:0:f48e:dbff:fe18:70e9%6:56828->/fe80:0:0:0:f48e:dbff:fe18:70e9%anpi0:5900, qualifier=null, endpoint=[fe80:0:0:0:f48e:dbff:fe18:70e9]:5900, alive=false, connectionType=NONE, planeIndex=-1] closed. Reason: Exception in Connection[id=2, /fe80:0:0:0:f48e:dbff:fe18:70e9%6:56828->/fe80:0:0:0:f48e:dbff:fe18:70e9%anpi0:5900, qualifier=null, endpoint=[fe80:0:0:0:f48e:dbff:fe18:70e9]:5900, alive=true, connectionType=NONE, planeIndex=-1], thread=hz.compassionate_bell.IO.thread-in-1
java.lang.IllegalStateException: Unknown protocol: RFB
    at com.hazelcast.internal.server.tcp.UnifiedProtocolDecoder.onRead(UnifiedProtocolDecoder.java:117)
    at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:138)
    at com.hazelcast.internal.networking.nio.NioPipeline.lambda$start$0(NioPipeline.java:127)
    at com.hazelcast.internal.networking.nio.NioThread.processTaskQueue(NioThread.java:355)
    at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:290)
    at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249)
    at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
TheOnlyAl commented 4 months ago

Very strange. Seems that this could be a problem with an already mapped port of 5900 and hazelcast for the embedded payara . If you can check if this port is already mapped and disable it, it might work, otherwise i could try to check if i can find out how to disable it.

TheOnlyAl commented 4 months ago

Hi @starksm64 i pushed an update where i changed the default hazelcast port to 15900. Seems there is a bug with some screen sharing software which reserves the port: https://github.com/payara/Payara/issues/3995

starksm64 commented 4 months ago

Ok, got it

uvula commented 4 months ago

Sorry, can I ask a question:

I'm just a arquillian user in my project and not so deep in the internals.

Can you please have a look to the following problem: https://stackoverflow.com/questions/78016093/how-can-i-control-transaction-handling-in-arquillian-using-junit5

Is this maybe related, so I can't control the transaction handling with junit5. Thanks

TheOnlyAl commented 4 months ago

Hi @uvula. I think your problem is not related to this problem since you do not seem to use https://github.com/arquillian/arquillian-extension-transaction/ as a dependency. If you would, your other tests would probably also fail since all transaction would roll back with this problem.

uvula commented 4 months ago

Hello @TheOnlyAl, I was also dealing with the dependencies:

Gradle:

    testFixturesApi 'org.jboss.arquillian.extension:arquillian-transaction-bom:1.0.6'
    testFixturesApi 'org.jboss.arquillian.extension:arquillian-transaction-api:1.0.6'
    testFixturesApi 'org.jboss.arquillian.extension:arquillian-transaction-jta:1.0.6'

arquillian.xml:

    <extension qualifier="transaction">
    <property name="manager">testManagerName</property>
    <property name="transactionDefaultMode">DISABLED</property>
  </extension>

and/or annotations @Transactional(value = TransactionMode.DISABLED) at class/method level. I don't have a running example with Junit4 (yet), so I can't check: DO I HAVE control with TransactionMode.*, somehow for my problem. The idea is to use DISABLED (see below).

My Junit tests are NOT failing. But I see, after calling a method of the stateless bean (without @javax.transaction.Transactional), asking entityManager em.isJoinedToTransaction() -> true. My understanding: subsequent method calls WITH @Transactional will join. So Envers can't see changes in between, just the final state after commit (wrapped by Arquillian/JUnit5)

Any hints? We also can use german.