vaadin / testbench

Vaadin TestBench is a tool for automated user interface testing of Vaadin applications.
https://vaadin.com/testbench
Other
20 stars 22 forks source link

@BrowserConfiguration not working with JUnit 5 @Nested annotation #1770

Open mcollovati opened 8 months ago

mcollovati commented 8 months ago

Currently, Testbench @BrowserConfiguration does not work correctly with JUnit @Nested annotation that is meant to work with not-static inner classes.

The problem is that when BrowserExtension.evaluateExecutionCondition evaluates whether the test should run or not, it tries to compute the requested capabilities; if the test class has a method annotated with @BrowserConfiguration, that method is supposed to be invoked to get the list of capabilities. However, when BrowserExtension.evaluateExecutionCondition is called, the JUnit test context does not yet have a reference to the test class instance (e.g. ExtensionContext.getTestInstance()), so Testbench tries to create a new instance by reflection calling the default no-args constructor, but it fails because it does not exist (inner classes always take the enclosing class instance as parameter).

To reproduce run the following test, that will fail because the @BrowserConfiguration gets not called

public class MyTest {

    @Nested
    class MyNested extends BrowserTestBase {

        private boolean browserConfigInvoked;

        @BrowserConfiguration
        public List<DesiredCapabilities> getBrowserConfiguration() {
            browserConfigInvoked = true;
            return List.of(BrowserUtil.chrome());
        }

        @BrowserTest
        void simpleTest() {
            Assertions.assertTrue(browserConfigInvoked,
                    "Expecting getBrowserConfiguration() to be invoked");
        }
    }
}

If you do the same on a top-level class, it works

public class MyTest extends BrowserTestBase  {

    private boolean browserConfigInvoked;

    @BrowserConfiguration
    public List<DesiredCapabilities> getBrowserConfiguration() {
        browserConfigInvoked = true;
        return List.of(BrowserUtil.chrome());
    }

    @BrowserTest
    void simpleTest() {
        Assertions.assertTrue(browserConfigInvoked,
                "Expecting getBrowserConfiguration() to be invoked");
    }
}