search

serenity-bdd / serenity-core

Serenity BDD is a test automation library designed to make writing automated acceptance tests easier, and more fun.
http://serenity-bdd.info
Other
718 stars 515 forks source link

Parallel execution is not working with cucumber 6 #2694

Closed jurec72 closed 2 years ago

jurec72 commented 2 years ago

Hi, please help to figure out how to run serenity with cucumber 6 in parallel. Try different ways and try to follow this link SERENITY CUCUMBER 6 PARALLEL, but still no result. Thank you for any suggestions!

Here is my pom.xml file:

`<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

4.0.0 
<groupId>com.d</groupId>
<artifactId>Automation</artifactId>
<version>1.0-SNAPSHOT</version>

<properties>
    <serenity.version>3.1.2</serenity.version>
    <serenity.cucumber.version>2.6.0</serenity.cucumber.version>
    <serenity.maven.version>3.1.2</serenity.maven.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <encoding>UTF-8</encoding>
    <tags></tags>
    <parallel.tests>4</parallel.tests>
    <webdriver.base.url></webdriver.base.url>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <java.version>11</java.version>
</properties>

<dependencies>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-rest-assured</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-cucumber6</artifactId>
        <version>${serenity.cucumber.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-junit5</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>com.googlecode.lambdaj</groupId>
        <artifactId>lambdaj</artifactId>
        <version>2.3.3</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.12.4</version>
    </dependency>
    <dependency>
        <groupId>javax.ws.rs</groupId>
        <artifactId>javax.ws.rs-api</artifactId>
        <version>2.1.1</version>
    </dependency>
    <dependency>
        <groupId>org.codehaus.jettison</groupId>
        <artifactId>jettison</artifactId>
        <version>1.4.1</version>
    </dependency>
    <dependency>
        <groupId>commons-dbutils</groupId>
        <artifactId>commons-dbutils</artifactId>
        <version>1.7</version>
    </dependency>
    <dependency>
        <groupId>com.microsoft.sqlserver</groupId>
        <artifactId>mssql-jdbc</artifactId>
        <version>9.4.0.jre11</version>
    </dependency>
    <dependency>
        <groupId>com.jayway.jsonpath</groupId>
        <artifactId>json-path</artifactId>
        <version>2.6.0</version>
    </dependency>
    <dependency>
        <groupId>com.github.javafaker</groupId>
        <artifactId>javafaker</artifactId>
        <version>1.0.2</version>
    </dependency>
    <dependency>
        <groupId>org.awaitility</groupId>
        <artifactId>awaitility</artifactId>
        <version>4.1.1</version>
    </dependency>

</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <skip>true</skip>
            </configuration>
        </plugin>
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <includes>
                    <include>**/*Runner.java</include>
                </includes>
                <systemPropertyVariables>
                    <webdriver.base.url>${webdriver.base.url}</webdriver.base.url>
                </systemPropertyVariables>
                <parallel>methods</parallel>
                <useUnlimitedThreads>true</useUnlimitedThreads>
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.1</version>
            <configuration>
                <source>11</source>
                <target>11</target>
                <compilerArgs>
                    <arg>-parameters</arg>
                </compilerArgs>
            </configuration>
        </plugin>
        <plugin>
            <groupId>net.serenity-bdd.maven.plugins</groupId>
            <artifactId>serenity-maven-plugin</artifactId>
            <version>${serenity.maven.version}</version>
            <configuration>
                <tags>${tags}</tags>
                <reports>single-page-html</reports>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>net.serenity-bdd</groupId>
                    <artifactId>serenity-single-page-report</artifactId>
                    <version>${serenity.version}</version>
                </dependency>
            </dependencies>
            <executions>
                <execution>
                    <id>serenity-reports</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>aggregate</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

`

wakaleo commented 2 years ago

The approach described in https://cucumber.io/docs/guides/parallel-execution/#junit-4 works. However you can't mix Serenity versions like that: make sure the Serenity dependencies all have the same version.

jurec72 commented 2 years ago

The approach described in https://cucumber.io/docs/guides/parallel-execution/#junit-4 works. However you can't mix Serenity versions like that: make sure the Serenity dependencies all have the same version.

Does it mean that I need downgrade all the serenity version to cucumber6 version? Because max version for cucumber 6 is 2.6.0, but serenity 3.1.2! Thank you!

wakaleo commented 2 years ago

Just use Cucumber 7.

jurec72 commented 2 years ago

Thank you for your suggestions. But still missing something. I try to run the same tag in different features, but still open one browser at a time. I use command: mvn clean verify And in runner put that tag! Thank you!

`<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

4.0.0 
<groupId>com.d</groupId>
<artifactId>Automation</artifactId>
<version>1.0-SNAPSHOT</version>

<properties>
    <serenity.version>3.1.20</serenity.version>
    <serenity.cucumber.version>3.1.20</serenity.cucumber.version>
    <serenity.maven.version>3.1.20</serenity.maven.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <encoding>UTF-8</encoding>
    <tags></tags>
    <parallel.tests>4</parallel.tests>
    <webdriver.base.url></webdriver.base.url>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <java.version>11</java.version>
</properties>

<dependencies>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-rest-assured</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-cucumber</artifactId>
        <version>${serenity.cucumber.version}</version>
    </dependency>
    <dependency>
        <groupId>org.junit.platform</groupId>
        <artifactId>junit-platform-launcher</artifactId>
        <version>1.8.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>5.8.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.vintage</groupId>
        <artifactId>junit-vintage-engine</artifactId>
        <version>5.8.2</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>com.googlecode.lambdaj</groupId>
        <artifactId>lambdaj</artifactId>
        <version>2.3.3</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.12.4</version>
    </dependency>
    <dependency>
        <groupId>javax.ws.rs</groupId>
        <artifactId>javax.ws.rs-api</artifactId>
        <version>2.1.1</version>
    </dependency>
    <dependency>
        <groupId>org.codehaus.jettison</groupId>
        <artifactId>jettison</artifactId>
        <version>1.4.1</version>
    </dependency>
    <dependency>
        <groupId>commons-dbutils</groupId>
        <artifactId>commons-dbutils</artifactId>
        <version>1.7</version>
    </dependency>
    <dependency>
        <groupId>com.microsoft.sqlserver</groupId>
        <artifactId>mssql-jdbc</artifactId>
        <version>9.4.0.jre11</version>
    </dependency>
    <dependency>
        <groupId>com.jayway.jsonpath</groupId>
        <artifactId>json-path</artifactId>
        <version>2.6.0</version>
    </dependency>
    <dependency>
        <groupId>com.github.javafaker</groupId>
        <artifactId>javafaker</artifactId>
        <version>1.0.2</version>
    </dependency>
    <dependency>
        <groupId>org.awaitility</groupId>
        <artifactId>awaitility</artifactId>
        <version>4.1.1</version>
    </dependency>

</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <skip>true</skip>
            </configuration>
        </plugin>
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <includes>
                    <include>**/*Runner.java</include>
                </includes>
                <systemPropertyVariables>
                    <webdriver.base.url>${webdriver.base.url}</webdriver.base.url>
                </systemPropertyVariables>
                <parallel>methods</parallel>
               <useUnlimitedThreads>true</useUnlimitedThreads>
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.1</version>
            <configuration>
                <source>11</source>
                <target>11</target>
                <compilerArgs>
                    <arg>-parameters</arg>
                </compilerArgs>
            </configuration>
        </plugin>
        <plugin>
            <groupId>net.serenity-bdd.maven.plugins</groupId>
            <artifactId>serenity-maven-plugin</artifactId>
            <version>${serenity.maven.version}</version>
            <configuration>
                <tags>${tags}</tags>
                <reports>single-page-html</reports>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>net.serenity-bdd</groupId>
                    <artifactId>serenity-single-page-report</artifactId>
                    <version>${serenity.version}</version>
                </dependency>
            </dependencies>
            <executions>
                <execution>
                    <id>serenity-reports</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>aggregate</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

`

jurec72 commented 2 years ago

Thank you, just found the solution and now parallel execution is working. If somebody need this, please check working pom.xml with serenity cucumber 7 and junit 4.

Working pom.xml: `<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

4.0.0 
<groupId>com.d</groupId>
<artifactId>Automation</artifactId>
<version>1.0-SNAPSHOT</version>

<properties>
    <serenity.version>3.2.0</serenity.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <encoding>UTF-8</encoding>
    <tags></tags>
    <parallel.tests>5</parallel.tests>
    <webdriver.base.url></webdriver.base.url>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
    <java.version>11</java.version>
</properties>
<dependencies>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-rest-assured</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-cucumber</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-junit</artifactId>
        <version>${serenity.version}</version>
    </dependency>
    <dependency>
        <groupId>com.googlecode.lambdaj</groupId>
        <artifactId>lambdaj</artifactId>
        <version>2.3.3</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.12.4</version>
    </dependency>
    <dependency>
        <groupId>javax.ws.rs</groupId>
        <artifactId>javax.ws.rs-api</artifactId>
        <version>2.1.1</version>
    </dependency>
    <dependency>
        <groupId>org.codehaus.jettison</groupId>
        <artifactId>jettison</artifactId>
        <version>1.4.1</version>
    </dependency>
    <dependency>
        <groupId>commons-dbutils</groupId>
        <artifactId>commons-dbutils</artifactId>
        <version>1.7</version>
    </dependency>
    <dependency>
        <groupId>com.microsoft.sqlserver</groupId>
        <artifactId>mssql-jdbc</artifactId>
        <version>9.4.0.jre11</version>
    </dependency>
    <dependency>
        <groupId>com.jayway.jsonpath</groupId>
        <artifactId>json-path</artifactId>
        <version>2.6.0</version>
    </dependency>
    <dependency>
        <groupId>com.github.javafaker</groupId>
        <artifactId>javafaker</artifactId>
        <version>1.0.2</version>
    </dependency>
    <dependency>
        <groupId>org.awaitility</groupId>
        <artifactId>awaitility</artifactId>
        <version>4.1.1</version>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <skip>true</skip>
            </configuration>
        </plugin>
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>3.0.0-M5</version>
            <configuration>
                <includes>
                    <include>**/*Runner.java</include>
                </includes>
                <systemPropertyVariables>
                    <webdriver.base.url>${webdriver.base.url}</webdriver.base.url>
                </systemPropertyVariables>
                <parallel>methods</parallel>
                <threadCount>${parallel.tests}</threadCount>
                <!-- <forkCount>${parallel.tests}</forkCount>-->
                <!--<useUnlimitedThreads>true</useUnlimitedThreads>-->
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.10.0</version>
            <configuration>
                <source>11</source>
                <target>11</target>
                <compilerArgs>
                    <arg>-parameters</arg>
                </compilerArgs>
            </configuration>
        </plugin>
        <plugin>
            <groupId>net.serenity-bdd.maven.plugins</groupId>
            <artifactId>serenity-maven-plugin</artifactId>
            <version>${serenity.version}</version>
            <configuration>
                <tags>${tags}</tags>
                <reports>single-page-html</reports>
            </configuration>
            <executions>
                <execution>
                    <id>serenity-reports</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>aggregate</goal>
                    </goals>
                </execution>
            </executions>
            <dependencies>
                <dependency>
                    <groupId>net.serenity-bdd</groupId>
                    <artifactId>serenity-single-page-report</artifactId>
                    <version>${serenity.version}</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

`

SaikatDEV commented 1 year ago

I have added this POM But i am having issues like " [INFO] --- maven-failsafe-plugin:3.0.0-M5:integration-test (default) @ serenity-test-automation --- [INFO] No tests to run. "

I have also tried to junit5, I am having the same error "No tests to run". I am using this mvn command to run : mvn clean verify -Dcucumber.filter.tags="@smoke" -Dwebdriver.driver=chrome serenity:aggregate

looks like, It couldn't find any tests to run. Not sure why. How to resolve this?

jurec72 commented 1 year ago

@SaikatDEV You can update everything to latest versions. Also, use command to run tests: mvn clean verify -Dcucumber.filter.tags="@smoke" -Dwebdriver.driver=chrome or try this command: mvn clean verify -D"cucumber.filter.tags"="@smoke" -D"webdriver.driver"=chrome Make sure that in the failsafe plugin you have the same name of the runner class. (*/Runner.java). I use pretty the same, but with latest versions and with serenity-core dependency and all working.

sarikakotha commented 1 year ago

This approach works and pics up execution- but when i set the thread count to 2 it is picking up 8 threads at once- After some scenarios its is picking up 2 threads and slowly execution happens in only one thread is there a way to control the threads- please suggest.

wakaleo commented 1 year ago

Cucumber does not currently support limiting the maximum number of threads, though there are properties to do so, they are not guaranteed to work (see https://github.com/cucumber/cucumber-jvm/tree/main/cucumber-junit-platform-engine#parallel-execution).

sarikakotha commented 1 year ago

okay- figured out the thread count is currently equal to number of feature files, i will try and see- Thanks much!