wttech / bobcat

Bobcat is an automated testing framework for functional testing of web applications.
https://cognifide.github.io/bobcat/
Apache License 2.0
90 stars 40 forks source link

[HELP] how to execute in parallel ? #207

Closed DeChrish closed 7 years ago

DeChrish commented 7 years ago

Need help to execute bdd feature files in parallel junit / mvn executor.

ghost commented 7 years ago

Hi,

Use maven-surefire-plugin. My working example:

<plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.16</version>
                <configuration>
                    <forkCount>${fork.count}</forkCount>
                    <reuseForks>false</reuseForks>
                    <includes>
                        <include>**/**/*${runner}.class</include>
                    </includes>
                </configuration>
            </plugin>
            <plugin>

The plugin will run classes with the name specified in ${runner} variable. Then you have to create classes which will run your tests. My example:

capture

As you can see, maven-surefire-plugin will run 5 classes in parallel. Each runner class has @tags to run a different set of tests.

Hope it helps

DeChrish commented 7 years ago

Hi, Thanks @F3Ni0 . It helps for parallel execution, but the problem here is it executes based on runner class.

Scenario: We have @regression tag added to 20 tests. How to run test in parallel with 5 browser instance?

  1. One Tag 2. One Test runner class
ghost commented 7 years ago

If you want to run 5 browser instances you need to create 5 runners (like in my screenshot). Tests can have multiple tags. You can tag them just for Runner's sake.

The example would be: @Regression tag for the whole Feature

@Regression <- this tag is applied to every scenario in this file
Feature: My feature

    @firstRunner <- this tag is applied only for this scenario
    Scenario: My scenario

    @firstRunner
    Scenario: My-another-scenario-for-the-same-runner

    @secondRunner
    Scenario: My scenario2

Label each scenario with some random tag and then, create 5 different runners like this:

capture

mkrzyzanowski commented 7 years ago

Hi @DeChrish,

unfortunately, these are the limitations of CucumberJVM. @F3Ni0's answer is what we also usually do for parallelization. You can also add the following plugin: https://github.com/temyers/cucumber-jvm-parallel-plugin - I've added it today in my project and seems like a way easier solution to this problem. It requires a bit of configuration but allows you to achieve your goal without the need to create separate runners. I'm going to test it a bit but for now, it looks like in future versions of projects generated from BDD archetype, it will be the preferred option in Bobcat: #209

DeChrish commented 7 years ago

@marcinczeczko Thanks a lot for #209.

DeChrish commented 7 years ago

@F3Ni0 i have created 3 Test Runner class. And added these class in maven-surefire-plugin pom. But the execution happens for only one Test Runner.

Found fix for the above issue,

Now i am not able to connect with implemented step definition. You can implement missing steps with the snippets below: But the missing steps are implemented.

ghost commented 7 years ago

@DeChrish I need more information in order to help you. Copy-paste your maven-surefire-plugin configuration, Test Runner class names, Screenshots with what's inside Test Runner classes. That's for start :)

DeChrish commented 7 years ago

@F3Ni0 was adding wrong glue path. Working now 👍

DeChrish commented 7 years ago

@F3Ni0 do we have gitter room for discussion?

ghost commented 7 years ago

@DeChrish Please, address your issues here. I am not a member of Bobcat team. I am just a user and I might not be able to answer all of your questions.

DeChrish commented 7 years ago

@F3Ni0 thanks ton for the help. closing this issue, tracking parallel execution framework fix in #209

sreedharsaravanan commented 6 years ago

@F3Ni0 , can you please let me know how you fixed execution happens for only one Test Runner. i have 18 runner file, but only one runner is executed and with one browser instance. below is my pom <plugin>

<groupId>net.masterthought</groupId> <artifactId>maven-cucumber-reporting</artifactId> <version>3.16.0</version> <executions> <execution> <id>execution</id> <phase>test</phase> <goals> <goal>generate</goal> </goals> <configuration> <projectName>bobcat</projectName> <outputDirectory>target/reports/cucumber-reports</outputDirectory> <cucumberOutput>target/cucumber-parallel/json</cucumberOutput>

                 `</configuration>`
                `</execution>`
            `</executions>`
        `</plugin>`
        `<plugin>`
            `<groupId>com.github.temyers</groupId>`
            `<artifactId>cucumber-jvm-parallel-plugin</artifactId>`
            <version>5.0.0</version>
            `<executions>`
                `<execution>`
                    `<id>generateRunners</id>`
                    `<phase>generate-test-sources</phase>`
                    `<goals>`
                        `<goal>generateRunners</goal>`
                    `</goals>`
                    `<configuration>`
                        `<glue>bobcat</glue>`
                        `<outputDirectory>src/main/java/autoGenerateRunner</outputDirectory>` <!-- runner will generate here -->
                        `<featuresDirectory>src/main/features/components</featuresDirectory><!--Cucumber` feature file location-->
                        `<cucumberOutputDir>target/cucumber-parallel</cucumberOutputDir><!--Cucumber` default reporting path-->
                        `<plugins>`
                            `<plugin>`
                                <!--The available options are junit, testng, html, pretty, json, usage and rerun -->
                                `<name>json</name>`
                                `<extension>json</extension>`
                                <!--Optional output directory. Overrides cucumberOutputDirectory. Usefull when different plugins create files with the same extension-->
                                `<outputDirectory>${project.build.directory}/cucumber-parallel/json/</outputDirectory>`
                            `</plugin>`
                        `</plugins>`
                        <strict>true<```

/strict>

true
                        <tags>
                            <!--<tag>@MultiplyNumbers,@AddNumbers</tag>-->
                                                <tag>${tag}</tag>
                        </tags>
                        <useTestNG>false</useTestNG>
                        <!-- The naming scheme to use for the generated test classes.  One of ['simple', 'feature-title', 'pattern'] -->
                        <namingScheme>simple</namingScheme>
                        <!-- The class naming pattern to use.  Only required/used if naming scheme is 'pattern'.-->
                        <namingPattern>Parallel{c}IT</namingPattern>
                        <!-- One of [SCENARIO, FEATURE]. SCENARIO generates one runner per scenario.  FEATURE generates a runner per feature. -->
                        <parallelScheme>FEATURE</parallelScheme>
                    </configuration>
                </execution>
            </executions>
        </plugin>

`<plugin>`
                `<artifactId>maven-compiler-plugin</artifactId>`
                `<version>3.3</version>`
                `<configuration>`
                    `<fork>true</fork>`
                    `<source>1.8</source>`
                    `<target>1.8</target>`
                    `<compilerVersion>1.8</compilerVersion>`
                `</configuration>`
            `</plugin>`
            `<plugin>`

            `</plugin>`
            `<plugin>`
                `<groupId>org.apache.maven.plugins</groupId>`
                `<artifactId>maven-surefire-plugin</artifactId>`
                `<version>2.18.1</version>`
                `<configuration>`
                    `<systemPropertyVariables>`
                        `<environment>${env}</environment>`
                        `<webdriver.type>${driverType}</webdriver.type>`
                        `<webdriver.url>${seleniumURL}</webdriver.url>`
                        `<webdriver.cap.browserName>${browserName}</webdriver.cap.browserName>`
                        <author.url>${aut```
horURL}</author.url>
                        <author.login>${authorLogin}</author.login>
                        <author.password>${authorPassword}</author.password>
                        <publish.url>${publishURL}</publish.url>
                        <publish.login>${publishLogin}</publish.login>
                        <publish.password>${publishPassword}</publish.password>
                        <configuration.pa
```ths>${configPath}</configuration.paths>
                    `</systemPropertyVariables>`
                    `<testFailureIgnore>true</testFailureIgnore>`
                    `<forkCount>10</forkCount>`
                    `<reuseForks>true</reuseForks>`
                    `<includes>`
                        <!--<include>**/**/*${runner}.class</include>-->
                        `<include>**/Parallel*IT.class</include>`
                    `</includes>`
                `</configuration>`
            `</plugin>`
`
ghost commented 6 years ago

@sreedharsaravanan You added everything except the runners. Add a screenshot.