No test are shown when all tests passed with forked unit tests

[Eccenux]

Apache NetBeans version

Apache NetBeans 22

What happened

I have some unit tests (org.junit.jupiter.api.Test). I execute tests, but when they are all OK (all passed) the I see status which indicates no testes where execute at all:

Only when I go to Output tab I can see:

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

Language / Project Type / NetBeans Component

Java Maven Spring application

How to reproduce

1. Create jupiter unit test file.
2. Add testFail.
3. Execute file: CTRL+F6.
4. -> "4 tests passed, 1 test failed" (OK)
5. Fix (uncomment fix).
6. Execute file: CTRL+F6.
7. -> "No tests executed" (bug)

Example test failing.

    @Test
    void testFail() {
        String input = "abc";
        String expected = "123";
//      // fix
//      input = expected;
        assertEquals(expected, input);
    }

Did this work correctly in an earlier version?

No / Don't know

Operating System

Windows 11

JDK

17

Apache NetBeans packaging

Apache NetBeans provided installer

Anything else

No response

Are you willing to submit a pull request?

No

[mbien]

you are saying if you run this, the window will say no tests executed? Can you try to reproduce this in a new project?

import org.junit.jupiter.api.Test;

public class Mavenproject1Test {
    @Test
    public void test1() {
    }
    @Test
    public void test2() {
    }
}

its working for me. Tested with junit.jupiter 5.11.0.

[Eccenux]

Interesting... I cannot reproduce it in a small, new project either... But I also have no idea where I should look for problems in the bigger project? I know the big project works fine in IntelliJ.

The test section of the output looks pretty much the same:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running pl.enux.maventestproject.JupiterNbTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.052 s -- in pl.enux.maventestproject.JupiterNbTest

Results:

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

The big project:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running pl.mol.molnet.opac.search.SlugTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 s -- in pl.mol.molnet.opac.search.SlugTest

Results:

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

Does NetBeans check the output to determine which tests passed, or does it use something else?

[matthiasblaesing]

@Eccenux if I remember correctly the code you are looking for is here:

https://github.com/apache/netbeans/blob/master/java/maven.junit/src/org/netbeans/modules/maven/junit/JUnitOutputListenerProvider.java

The code uses a combination of "listening" to the output and reading the surefire/failsafe XML outputs. If the stdout output looks identical, the assumption would be, that the XML outputs are not at the expected location or maybe surpressed?

[mbien]

@Eccenux would you like to give this dev-build a try https://github.com/apache/netbeans/actions/runs/10692992128/artifacts/1888805200?

this is addressing an issue with multi-module builds and the test window - maybe we are lucky and it fixes your problem too.

[Eccenux]

Dev build didn't help. But I was able to reproduce.

Here is my small test project with which you should be able to reproduce the problem (also in the dev build): https://github.com/Eccenux/MavenTestProject/blob/master/pom.xml#L74

It's a problem with <tests.unit.forkCount>2</tests.unit.forkCount>. When modified to 1 fork it works as expected, 2 forks and it brakes.

[mbien]

@Eccenux thanks for testing and the info! I suspected that it might be due to fork or other concurrency related problems (parallel build or mvnd).

The detection for this situation is not very good right now, it will also fail if the config for this is in the parent pom.

I am wondering if we should try a different approach there, e.g adding a file watch to the folder instead of delaying the check for the xml files.