search

prashant-ramcharan / courgette-jvm

Multiprocess | Parallel Cucumber-JVM | Parallelize your Java Cucumber tests on a feature level or on a scenario level.
MIT License
130 stars 38 forks source link
appium automated-testing courgette-jvm courgette-report courgette-runner cucumber cucumber-features cucumber-jvm failed-scenarios gherkin mobile-automation mobile-testing parallel parallel-cucumber-jvm selenium test-automation

readme

Maven Central License: MIT

Courgette-JVM

Courgette-JVM is an extension of Cucumber-JVM with added capabilities to run cucumber tests in parallel on a feature level or on a scenario level. It also provides an option to automatically re-run failed scenarios.

Key Features

Minimum Requirements

Installation

Maven

<dependency>
  <groupId>io.github.prashant-ramcharan</groupId>
  <artifactId>courgette-jvm</artifactId>
  <version>6.14.0</version>
</dependency>

Gradle

implementation group: 'io.github.prashant-ramcharan', name: 'courgette-jvm', version: '6.14.0'

Included Cucumber Dependencies

Usage

Example projects:

Courgette-JVM supports JUnit and TestNG to run cucumber features and scenarios in parallel. A JUnit runner class must be annotated with @RunWith(Courgette.class) and a TestNG runner class must extend TestNGCourgette.

Additional

JUnit Runner
@RunWith(Courgette.class)
@CourgetteOptions(
        threads = 10,
        runLevel = CourgetteRunLevel.SCENARIO,
        rerunFailedScenarios = true,
        rerunAttempts = 1,
        testOutput = CourgetteTestOutput.CONSOLE,
        reportTitle = "Courgette-JVM Example",
        reportTargetDir = "build",
        environmentInfo = "browser=chrome; git_branch=master",
        cucumberOptions = @CucumberOptions(
                features = "src/test/resources/features",
                glue = "steps",
                tags = "@regression and not @bug",
                publish = true,
                plugin = {
                        "pretty",
                        "json:build/cucumber-report/cucumber.json",
                        "html:build/cucumber-report/cucumber.html",
                        "junit:build/cucumber-report/cucumber.xml"}
        ))
public class RegressionTestSuite {
}
TestNG Runner
@Test
@CourgetteOptions(
        threads = 10,
        runLevel = CourgetteRunLevel.SCENARIO,
        rerunFailedScenarios = true,
        rerunAttempts = 1,
        testOutput = CourgetteTestOutput.CONSOLE,
        reportTitle = "Courgette-JVM Example",
        reportTargetDir = "build",
        environmentInfo = "browser=chrome; git_branch=master",
        cucumberOptions = @CucumberOptions(
                features = "src/test/resources/features",
                glue = "steps",
                tags = "@regression and not @bug",
                publish = true,
                plugin = {
                        "pretty",
                        "json:build/cucumber-report/cucumber.json",
                        "html:build/cucumber-report/cucumber.html"}
        ))
public class RegressionTestSuite extends TestNGCourgette {
}

Gradle Build Task

tasks.withType(Test) {
    systemProperties = System.getProperties()
}

// JUnit
task regressionSuite(type: Test) {
    include '**/RegressionTestSuite.class'
    outputs.upToDateWhen { false }
}

// TestNG
task regressionSuite(type: Test) {
    useTestNG()
    include '**/RegressionTestSuite.class'
    outputs.upToDateWhen { false }
}

Override Runner Options

To override the hard-coded courgette options (threads, runLevel, rerunFailedScenarios, reportTargetDir, environmentInfo) set in the runner class, you can provide system properties to the gradle or maven task.


[gradle | mvn] test -Dcourgette.threads=2 -Dcourgette.runLevel=FEATURE -Dcourgette.rerunFailedScenarios=false -Dcourgette.reportTargetDir=build -Dcourgette.environmentInfo="git_branch=master; platform=ci"

To override the hard-coded cucumber options (tags, glue, plugin, name, junit) set in the runner class, you can provide comma separated system properties to the gradle task.


[gradle | mvn] test -Dcucumber.tags="@regression and not @bug" -Dcucumber.glue="steps, hooks"

To specify non standard VM options (-X options)


[gradle | mvn] test -Dcourgette.vmoptions="-Xms256m -Xmx512m"

JUnit Callbacks

You can add global setup and tear-down code to your Courgette test runner using the @CourgetteBeforeAll and @CourgetteAfterAll annotations. For example:

@RunWith(Courgette.class)
@CourgetteOptions(/* Your Courgette options here... */)
public class RegressionTestSuite {
    @CourgetteBeforeAll
    public static void setUp() {
        System.out.println("I will run before any tests execute");
    }

    @CourgetteAfterAll
    public static void tearDown() {
        System.out.println("I will run after all of the tests execute");
    }
}

You can add any number of annotated methods to your test suite class. If you need your callbacks to run in a specific order, pass order to the annotation: @CourgetteBeforeAll(order = 2).

Courgette Run Information

You can access test statistics and additional run information if you need to analyze or perform extra tasks before or after the parallel test run.

Note: CourgetteRunInfo can only be accessed from a Courgette runner class.

JUnit Runner 

@RunWith(Courgette.class)
@CourgetteOptions(/* Your Courgette options here... */)
public class RegressionTestSuite {

    @CourgetteBeforeAll
    public static void beforeRun() {
        System.out.println("Starting Courgette parallel run: " + CourgetteRunInfo.sessionId());
    }

    @CourgetteAfterAll
    public static void afterRun() {
      if (CourgetteRunInfo.testStatistics().hasFailures()) {
        // do something extra here
      }
    }
}

TestNG Runner 

@Test
@CourgetteOptions(/* Your Courgette options here... */)
public class RegressionTestSuite extends TestNGCourgette {

    @BeforeTest
    public static void beforeRun() {
        System.out.println("Starting Courgette parallel run: " + CourgetteRunInfo.sessionId());
    }

    @AfterTest
    public static void afterRun() {
        if (CourgetteRunInfo.testStatistics().hasFailures()) {
            // do something extra here
        }
    }
}

Retrieve the Courgette Thread ID and Name

System.getProperty("courgette.threadId")

System.getProperty("courgette.threadName")

Slack Integration

Courgette allows real time test results and events to be posted to Slack as tests are run.

To enable this feature, add the following Courgette options to the Courgette runner:

@CourgetteOptions(
      ...
      slackWebhookUrl = "https://hooks.slack.com/services/your-slack-url",
      slackChannel = {"channel1", "channel2"},
      slackTestId = "Production test - Build 1.0.0",
      slackEventSubscription = {CourgetteEvent.ALL},
      cucumberOptions = @CucumberOptions(
      // cucumber options here
      )
)

Slack Incoming Webhook URL

You need to create an incoming webhook URL to allow Courgette to post messages to your Slack application.

https://api.slack.com/messaging/webhooks#create_a_webhook

Event Subscription

You can subscribe to single or multiple Courgette events. When events are triggered as the tests run, Courgette will post a message to the Slack channels defined in the runner.

CourgetteJVM_Slack.png

Report Portal Integration

Courgette allows test results to be published in real time to the Report Portal server as tests run.

To enable this feature, add the following Courgette option to the Courgette runner:

@CourgetteOptions(
      ...  
      plugin = { CourgettePlugin.REPORT_PORTAL },
      cucumberOptions = @CucumberOptions(
      // cucumber options here
      )
)

You must have the reportportal.properties file in your classpath and the following properties must be defined:

# Report Portal server (mandatory)
rp.endpoint = http://localhost:8080

# Report Portal project (mandatory)
rp.project = courgette_example

# Report Portal API access token (mandatory)
rp.apitoken = a1e5ee78-317c-477d-b27e-f174c562aedc

# Report Portal launch name (optional)
rp.launch = My Demo Project

# Report Portal test suite (optional)
rp.testsuite = Regression Test Suite

# Report Portal launch attributes (optional)
# Each attribute must be separated by ';'
rp.attributes = suite:regression;build:12345

Note: Any property other than those defined above will be ignored by Courgette.

An API access token is required to allow Courgette to publish the report. To obtain an API access token, log in to Report Portal UI and navigate to http://localhost:8080/ui/#api -> UAT -> sso-endpoint -> Get api token

After the test run is complete, the test results will be published to the Report Portal server.

CourgetteJVM_ReportPortal.png

Extent Reports Integration

Courgette allows the creation of interactive reports using the Extent Reports Courgette plugin.

To enable this feature, add the following Courgette option to the Courgette runner:

@CourgetteOptions(
      ...  
      plugin = { CourgettePlugin.EXTENT_REPORTS },
      cucumberOptions = @CucumberOptions(
      // cucumber options here
      )
)

At the end of the test run the report will be saved to ${reportTargetDir}/courgette-extentreports

To configure custom reports (i.e. change the report name or theme) you should create the extent-config.xml file in the classpath. Courgette will load this XML config when it builds the report. If this file is not provided then default values will be used. View an example here

CourgetteJVM_ExtentReports.png

Allure Integration

Courgette allows the generation of Allure reports using the Allure Cucumber plugin.

@CourgetteOptions(
      ...
      cucumberOptions = @CucumberOptions(
         plugin = {
            "io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm"
         }
      )
)

CourgetteJVM-AllureReport.png

Courgette Mobile Device Allocator

Courgette provides a mobile device allocator to allocate and keep track of devices for parallel mobile testing. Courgette keeps track of devices that are currently in use and automatically allocates a free device for each test.

@CourgetteOptions(
        ...
        plugin = { CourgettePlugin.MOBILE_DEVICE_ALLOCATOR },
        mobileDeviceType = MobileDeviceType.SIMULATOR,
        mobileDevice = {
                "iPhone 8",
                "iPhone 12",
                "iPhone 13"
        },
        cucumberOptions = @CucumberOptions(
        // cucumber options here
        )
)

The Courgette mobile device allocator plugin will:

Notes:

How to integrate Courgette Mobile Device Allocator

The above properties are only available when running tests using a Courgette runner with the CourgettePlugin.MOBILE_DEVICE_ALLOCATOR plugin.

iOS Appium Test Configuration

DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("deviceName", CourgetteMobileDeviceAllocator.DEVICE_NAME); 
capabilities.setCapability("udid", CourgetteMobileDeviceAllocator.UDID); 
capabilities.setCapability("wdaLocalPort", CourgetteMobileDeviceAllocator.PARALLEL_PORT);

Android Appium Test Configuration

DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("avd", CourgetteMobileDeviceAllocator.DEVICE_NAME); 
capabilities.setCapability("udid", CourgetteMobileDeviceAllocator.UDID); 
capabilities.setCapability("systemPort", CourgetteMobileDeviceAllocator.PARALLEL_PORT);

SauceLabs Appium Test Configuration

MutableCapabilities capabilities = new MutableCapabilities();
capabilities.setCapability("appium:deviceName", CourgetteMobileDeviceAllocator.DEVICE_NAME);

BrowserStack Appium Test Configuration

DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("device", CourgetteMobileDeviceAllocator.DEVICE_NAME);

Pass Mobile Devices at Runtime

[mvn | gradle] test -Dcourgette.mobileDevice="iPhone X, iPhone 12, iPhone 13"

Using Real Devices

Unlike simulators where the device name must be unique, on real devices you can specify the same device name as long as the UUIDs are different for each device.

Format: deviceName:deviceUUID

mobileDevice = {
     "iPhone 8:00000000-000-0000-0000-000000000001",
     "iPhone 8:00000000-000-0000-0000-000000000002",
     "iPhone 8:00000000-000-0000-0000-000000000003"
}

Limitations and Known Issues

Submitting Issues

For any issues or requests, please submit here