Pragmatists / JUnitParams

Parameterised tests that don't suck
http://pragmatists.github.io/JUnitParams
Apache License 2.0
888 stars 149 forks source link

Cannot exclude specific invocations from Filter #153

Open marcphilipp opened 6 years ago

marcphilipp commented 6 years ago

A Filter that is applied to a JUnitParamsRunner gets passed different Description instances than the runner actually uses.

Example

Running the main method from the following example demonstrates the issue.

@RunWith(JUnitParamsRunner.class)
public class JUnitParamsTestCase {
    @Test
    @Parameters({ "17, false", "22, true" })
    public void personIsAdult(int age, boolean valid) {
    }

    public static void main(String... args) throws Exception {

        Runner runner = Request.aClass(JUnitParamsTestCase.class).getRunner();

        System.out.println("Full Description:");
        printRecursively(runner.getDescription(), "");
        System.out.println();

        System.out.println("Filtering:");
        Filter filter = new Filter() {
            @Override
            public boolean shouldRun(Description description) {
                System.out.println("shouldRun? description=" + description + ", children=" + description.getChildren());
                return !description.getDisplayName().contains("17, false");
            }

            @Override
            public String describe() {
                return "exclude everything";
            }
        };
        filter.apply(runner);
        System.out.println();

        System.out.println("Running:");
        JUnitCore core = new JUnitCore();
        core.addListener(new RunListener() {
            @Override
            public void testFinished(Description description) {
                System.out.println("finished: " + description);
            }
        });
        core.run(runner);
    }

    private static void printRecursively(Description description, String indent) {
        System.out.println(indent + "- " + description);
        description.getChildren().forEach(c -> printRecursively(c, indent + "  "));
    }
}

Output

Full Description:
- org.junit.vintage.engine.samples.junit4.JUnitParamsTestCase
  - personIsAdult
    - personIsAdult(17, false) [0](org.junit.vintage.engine.samples.junit4.JUnitParamsTestCase)
    - personIsAdult(22, true) [1](org.junit.vintage.engine.samples.junit4.JUnitParamsTestCase)

Filtering:
shouldRun? description=personIsAdult(org.junit.vintage.engine.samples.junit4.JUnitParamsTestCase), children=[]
shouldRun? description=personIsAdult(org.junit.vintage.engine.samples.junit4.JUnitParamsTestCase), children=[]

Running:
finished: personIsAdult(17, false) [0](org.junit.vintage.engine.samples.junit4.JUnitParamsTestCase)
finished: personIsAdult(22, true) [1](org.junit.vintage.engine.samples.junit4.JUnitParamsTestCase)

Actual Behavior

The Description instances passed to the Filter are generated by describeChild and contain no children and no information about the parameters being used. Thus, it's impossible to exclude them based on this information.

Expected Behavior

The Filter receives the same Description instances used as direct children of the Description returned by JUnitParamsRunner.getDescription().

Potentially Related Issues

mmorrisontx commented 4 years ago

If anyone else hits this problem in the future, I can confirm that the patch in https://github.com/Pragmatists/JUnitParams/pull/123 does actually solve the problem.