Cucumber-spring/Selenium/Surefire - parallel running support

[jonn-set]

I am trying to use the Maven surefire plugin to run my Browser automation tests in parallel.

In my Test framework, I use the following:

• Cucumber 7
• Cucumber Spring for DI
• Selenium 4.18.1
• Junit 4 (CucumberOptions)
• Maven surefire for running running tests

I inject the WebDriver and I know the bean is a singleton instance unless I annotate it with @ScenarioScope , however when I annotate it with @ScenarioScope , it opens as many browser windows as my Test Scenarios. I have tried annotating it with @Lazy but that hasn't worked either.

I am only passing 2 threads in surefire, so was hoping only 2 tests to start.

Is it possible to limit the WebDriver injection to the number of threads or when the scenario is initiated and not for every scenario when test run start?

Any help is appreciated.

High level code:

Maven - surefire:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <executions>
        <execution>
            <id>first-run</id>
            <phase>integration-test</phase>
            <goals>
                <goal>test</goal>
            </goals>
            <configuration>
                <includes>
                    <include>**/*TestRunner.java</include>
                </includes>
                <testFailureIgnore>true</testFailureIgnore>
                <rerunFailingTestsCount>0</rerunFailingTestsCount>
                <parallel>methods</parallel>
                <threadCount>3</threadCount>
            </configuration>
        </execution>
    </executions>
</plugin>

WebDriver bean:

@Configuration
@ComponentScan(basePackages = "com.example")
public class DriverConfig {

  @Bean(name = "driver", destroyMethod = "quit")
  @ScenarioScope(proxyMode = ScopedProxyMode.NO) // have tried all available options here, none work
  public WebDriver getDriver() {
    EdgeOptions options = new EdgeOptions();
    options.addArguments(chromiumOptions());
    EdgeDriverService service =
          new EdgeDriverService.Builder()
              .withLoglevel(ChromiumDriverLogLevel.OFF)
              .build();
      return new EdgeDriver(service, options);
  }

WebDriver Injection:

WebDriver is Autowired where ever its needed and is a private member of the class.

@Autowired private WebDriver driver;

[mpkorstanje]

Interesting question but this isn't really a problem for Cucumber to solve. The design pattern you are looking for is called "Object Pool". You could use the Apache Commons Pool to do the heavy lifting.

Then it would look something like this.

@Configuration
public class TestConfiguration {

    @Bean
    GenericObjectPoolConfig<WebDriver> genericObjectPoolConfig(){
        var config = new GenericObjectPoolConfig<WebDriver>();

        // Externalize these settings to application-test.yaml
        // Though the defaults work probably fine, as long as your threads 
        // don't exceed the max desired drivers.
        config.setMaxTotal(2);
        config.setBlockWhenExhausted(true);
        return config;
    }

    @Bean
    PooledObjectFactory<WebDriver> webDriverPooledObjectFactory(){
        return new BasePooledObjectFactory<>() {
            @Override
            public WebDriver create() {
                return // create actual web driver here.
            }

            @Override
            public PooledObject<WebDriver> wrap(WebDriver o) {
                return new DefaultPooledObject<>(o);
            }

            @Override
            public void passivateObject(PooledObject<WebDriver> p) {
                System.out.println("Returned");
                // Reset web driver to defaults here for reuse in the next test
            }

            @Override
            public void destroyObject(PooledObject<WebDriver> p) {
                System.out.println("Destroyed");
                // Shutdown web driver here
            }
        };
    }

    @Bean
    ObjectPool<WebDriver> webDriverObjectPool(PooledObjectFactory<WebDriver> factory, GenericObjectPoolConfig<WebDriver> config){
        return new GenericObjectPool<>(factory, config);
    }

    @Bean
    @ScenarioScope // This will scope this bean to the scenarios life cycle.
    public PooledWebDriver webDriver(ObjectPool<WebDriver> pool) {
        // Inject this class into your step definitions
        return new PooledWebDriver(pool);
    }

public static class PooledWebDriver implements AutoCloseable {
    private final ObjectPool<WebDriver> pool;
    private WebDriver instance;
    public PooledWebDriver(ObjectPool<WebDriver> pool) {
        this.pool = pool;
    }

    public WebDriver getWebDriver() {
        if (instance == null) {
            try {
                // Lazy borrowing, in case the web driver isn't actually used.
                instance = pool.borrowObject();
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }
        return instance;
    }

    @Override
    public void close() {
        if (instance == null) {
            return;
        }
        try {
            // Because the PooledWebDriver bean is scoped to the scenario
            // life cycle the close method will be invoked and the web-driver instance returned.
            pool.returnObject(instance);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

The PooledWebDriver is a bit of a rough sketch. You could have it implement the WebDriver interface and call delegate all calls (except close) to the actual web driver instance. Then you can also inject WebDriver as you would normally.

[jonn-set]

Thanks @mpkorstanje, appreciate your detailed reply.

This looks a bit complex to implement and maintain. If I leave the project, other Test Engineers might struggle to understand what's happening. Is there no simpler way to do this?

I know this project is already running cucumber/spring tests parallelly. I would have used this library but it may be denied approval to use on our CI environments by my employer. And I haven't had time to look into the code fully to understand how things are done there.

[mpkorstanje]

Unfortunately, I don't think there is a simpler way to do this.

The complexity comes from 1) doing things in parallel and then 2) sharing a limited number of resources (the web drivers) between the parallel test executions. If you drop the first requirement, then you can use a singleton web driver. If you drop the second requirement then you can use the @ScenarioScope web driver.

You could hack things together with a ThreadLocal and that will look simpler at first. But then you're very much dependent on there being a constant number of threads (this not the case with JUnit 5 i.c.m the Cucumber JUnit Platform Engine) and the clean up of the web drivers gets quite difficult.

[mpkorstanje]

I know this project is already running cucumber/spring tests parallelly. And I haven't had time to look into the code fully to understand how things are done there.

You are implying that you have not been able to run Cucumber in parallel. Am I interpreting that correctly?

[jonn-set]

I know this project is already running cucumber/spring tests parallelly. And I haven't had time to look into the code fully to understand how things are done there.

You are implying that you have not been able to run Cucumber in parallel. Am I interpreting that correctly?

No, I am not implying I cannot run it in parallel, because I haven't tried running Cucumber on its own yet.

This library has got some examples in which they are running cucumber/spring tests in parallel, not sure if it's possible to run cucumber/spring/webdriver together in parallel using this library. In this library, the number of threads is passed as a '@Option' not through surefire plugin.

[mpkorstanje]

Ah I see. I haven't used that project and I can't really comment on it.